rokujyouhitoma has quit [Ping timeout: 260 seconds]
realitix has joined #pypy
Ubuntu-BR has joined #pypy
jwhisnant has quit [Ping timeout: 260 seconds]
jwhisnant has joined #pypy
vkirilichev has joined #pypy
<kenaan>
Raemi stmgc-c8 8d2c43276bbe /pypy/stm/print_stm_log.py: (tobweber) fix wrong thread state after waiting for free segment
<kenaan>
Raemi stmgc-c8 333cd7c29400 /rpython/jit/backend/llsupport/test/test_stmrewrite.py: add passing stmrewrite test
vkirilichev has quit [Ping timeout: 260 seconds]
rokujyouhitoma has joined #pypy
Taggnostr has joined #pypy
rokujyouhitoma has quit [Ping timeout: 268 seconds]
cstratak has joined #pypy
cstratak has quit [Remote host closed the connection]
<njs>
huh, I think I found a pretty serious bug in pypy's ssl module actually
<njs>
if the underlying socket call returns an error then it seems to never report that, and instead raise SSLEOFError
<LarstiQ>
eh
<LarstiQ>
s/?//
<LarstiQ>
njs: go on??
cstratak_ has joined #pypy
<njs>
(so not like an "oh no teh CVEs" kind of bug, but pretty bad in that it makes it impossible to tell the difference between a clean shutdown and a connection reset)
<njs>
but it turns out much simpler programs have the same issue
adamholmberg has quit [Ping timeout: 260 seconds]
<arigato>
I guess from your senario that you can come up with a pair of 2x5-lines reproducer
<arigato>
I must say I don't know much about ssl, but I'd guess that would be useful for plan_rich
<njs>
well, SSL reproducers tend to be a little longer than that because you need ~30 lines just to set up the connection :-)
<njs>
uh
<njs>
and when I try my script on pypy2, then it fails to even create an SSL socket object
<njs>
File "/home/njs/pypy/pypy-5.8-linux_x86_64-portable/lib-python/2.7/ssl.py", line 598, in __init__
<njs>
AttributeError: '_socket.socket' object has no attribute '_sock'
<njs>
socket.__init__(self, _sock=sock._sock)
arigato has quit [Ping timeout: 240 seconds]
arigato has joined #pypy
<njs>
okay yeah, I convinced read/write to return EINVAL, and the ssl module is reporting even that as SSLEOFError, which makes no sense at all
jacob22_ has joined #pypy
<njs>
plan_rich: arigato: updated that bug with a simplified reproducer
<njs>
oh the python2 thing is because... apparently it's expected that you can't SSL-wrap a socketpair() on python 2, because socketpair objects are a differnet type than regular sockets? python 2 is so weird.
rokujyouhitoma has joined #pypy
arigato has quit [Read error: Connection reset by peer]
Ubuntu-BR has quit [Ping timeout: 255 seconds]
arigato has joined #pypy
<arigato>
yes, python2's socket module is strange. python3's too, but less so I think
rokujyouhitoma has quit [Ping timeout: 246 seconds]
arigato has quit [Read error: No route to host]
squeaky_pl has joined #pypy
Ubuntu-BR has joined #pypy
arigato has joined #pypy
arigato has quit [Read error: Connection reset by peer]
<inad922>
How can I reliably access python objects from a c extension written with cffi?
<njs>
inad922: you cannot
<inad922>
I mean last time I asked I got the answer that I should not use PyObject from Python.h since that's CPython specific.
<inad922>
njs: That's bad.
<inad922>
Is there no workaround
<inad922>
?
<njs>
inad922: I find it a bit annoying too tbh
<inad922>
I mean I've written an AVL
<inad922>
it's really slow in python
<njs>
inad922: but not that I know of
<inad922>
Quite the opposite in c
<inad922>
And I would like to make it work in pypy in a "correct" way
<inad922>
It's actually also annoying that there are no bsts in python
<inad922>
I miss them time to time
<inad922>
njs: Alright I just wait for someone to pop up here with the answer then :)
<njs>
cfbolz: is it... expected that an unreferenced object could survive a call to gc.collect(), and then be collected later?
<njs>
cfbolz: I have a test that explicitly does gc.collect() to prevent "corouting was never awaited" warning messages popping up later, but they're still popping up later...
<njs>
inad922: the usual advice is to write a C library that does what you want, then use cffi to wrap it
<njs>
inad922: so your C code doesn't talk to python, only python talks to your C code
nimaje1 has joined #pypy
nimaje1 is now known as nimaje
nimaje has quit [Killed (orwell.freenode.net (Nickname regained by services))]
<inad922>
njs: Yeah, but the keys/values can be python objects. Also I have to modify the reference counter when the key/value is part of the tree.
<inad922>
njs: Or when I remove them from the tree.
<njs>
inad922: ...I guess if what you're trying to do is implement a container class then yeah, that's an issue. I don't know how to do that; it's quite tricky.
<cfbolz>
inad922: I don't think there is a good answer for that, honestly
<njs>
inad922: pypy works fundamentally differently from cpython in this regard; it has no reference counts, and it wants to know where every pointer-to-python-object is stored because sometimes it moves objects around and then rewrites the pointers to point to the new location
oberstet has quit [Ping timeout: 240 seconds]
oberstet2 has joined #pypy
<cfbolz>
njs: yes, objects with finalizers can survive several collects
<cfbolz>
particularly if they are in cycles with other objects with finalizers
<njs>
in principle I guess cffi could provide some sort of interface for boxing up a python object into an opaque C value and then deallocate it later, but I don't think it does.
<inad922>
ah
<cfbolz>
njs: it does
<njs>
cfbolz: oh?
<inad922>
Could you point me to the implementation of the list/set/dict in the pypy codebase?
<cfbolz>
inad922: that is written in rpython, not C
<inad922>
cfbolz: What is rpython? :)
<inad922>
Sorry I don't know that
<cfbolz>
the language that pypy is implemented in
<cfbolz>
it's a subset of python
<njs>
oh huh. well, using ffi.new_handle would be a bit annoying b/c of needing to keep the objects alive, but I guess it could solve inad922's problem
<cfbolz>
njs: yes, the "keeping things alive" part is the reason why containers are still not fully solved
<njs>
as long as they don't mind keeping a set() of handle objects alongside their tree, though I guess that might defeat the purpose :-)
<njs>
cfbolz: have you considered... refcounting
<cfbolz>
😒
<njs>
cfbolz: anyway, regarding my gc thing: that's odd, calling gc.collect() repeatedly does seem to help. I say it's odd because the object here is a coroutine object returned by 'async def f(): pass', which really shouldn't be in any cycle or be resurrecting itself I would think
oberstet3 has joined #pypy
oberstet2 has quit [Ping timeout: 240 seconds]
rokujyouhitoma has joined #pypy
<njs>
(...actually, a totally reasonable api for the handle thing would be to require the user explicitly pass them to a free() function. this is C after all...)
<inad922>
cfbolz: This new_handle/from_handle seems handy, but how can I call a comparison function on these?
<inad922>
In the C code obviusly
<inad922>
obviously*
<njs>
cfbolz: though, hmm, I guess the finalizer does need to touch self to get the coroutine name and put it in the message (and also just to run the __del__ frame); maybe pypy considers that to be a temporary resurrection
<cfbolz>
njs: basically if you are fine with such a "free_handle" interface, you can just build that yourself
<cfbolz>
(in python, with a set)
antocuni has quit [Ping timeout: 268 seconds]
<njs>
cfbolz: the problem is that I'm assuming inad922 wants to implement a new container because they consider set() to have some properties they don't like :-)
<njs>
cfbolz: like why implement an AVL tree unless you want bounded worst case insert/delete
oberstet3 has quit [Ping timeout: 258 seconds]
<cfbolz>
njs: yes I see
<inad922>
njs: Yep. Set is a hash. It's not ordered. For this reason some problems can't be solved with it. Say you have the following problem. You have 3 different operations. 1. insert integer in the data structure. 2. delete integer from the data structure 3. return the sum of elements from index i to index j. This can't be efficiently solved with set/list/dict but all operations run in O(log n) with an AVL tree.
<njs>
inad922: well, that *particular* case can be solved efficiently with a set+AVL tree, so I guess if that's the problem you care about then the handle interface is ok :-)
<inad922>
njs: Nah, that's not the reason I implemented this. I just wanted to give an example that is a good use case for a BST.
<cfbolz>
njs: cpyext solves this problem partly, so I suppose we should think how to have a similar approach in cffi
inad922 has quit [Ping timeout: 240 seconds]
jcea has joined #pypy
lritter has joined #pypy
<squeaky_pl>
What are the copying characteristics about passing unicode, bytestring and bytearrays in cffi these days?
amaury_ has joined #pypy
jacob22_ has quit [Quit: Konversation terminated!]
<squeaky_pl>
if you are lucky things can be "pinned"?
oberstet has joined #pypy
<dstufft>
njs: Yes it does, it has to download it on demand but yea
cstratak_ has quit [Quit: Leaving]
cstratak has joined #pypy
arigato has joined #pypy
nedbat has quit [Excess Flood]
nedbat has joined #pypy
Ubuntu-BR has quit [Ping timeout: 258 seconds]
marky1991_2 has joined #pypy
Ubuntu-BR has joined #pypy
<kenaan>
arigo default 9a5adcd4d3ff /rpython/jit/backend/x86/assembler.py: Avoid one instruction in the endless stream of header/footer instructions
rokujyouhitoma has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 240 seconds]
jacob22_ has joined #pypy
amaury_ has quit [Quit: Konversation terminated!]
amaury_ has joined #pypy
marky1991_2 has quit [Ping timeout: 255 seconds]
Rhy0lite has joined #pypy
<arigato>
another potential improvement source:
<arigato>
the blackhole interpreter's "getfield_gc_r" instruction, for example, expands to pages and pages of checks for sanity
<arigato>
I think I eventually found the actual "mov" instruction inside doing the load, but it took me a while
antocuni has joined #pypy
amaury_ has quit [Ping timeout: 246 seconds]
rokujyouhitoma has joined #pypy
yuyichao has quit [Ping timeout: 240 seconds]
rokujyouhitoma has quit [Ping timeout: 260 seconds]
<cfbolz>
arigato: hmm :-(
<cfbolz>
Did you take a look at the paper draft I sent you?
<cfbolz>
No need to read it all, looking at pictures is enough
<arigato>
yes
<arigato>
that's why I'm looking around
girish946 has joined #pypy
yuyichao has joined #pypy
<squeaky_pl>
I wish every book was like that, no reading needed just look at the pictures ;-)
<cfbolz>
arigato: ah, cool ;-)
<arigato>
cfbolz: do you know what jit instructions are categorized as "control flow"?
girish946 has quit [Quit: Leaving]
<arigato>
or "ctrl"
<cfbolz>
Heh, I had the same question, will forward you the answer
<arigato>
yes, though I didn't quite expect that the "*_return" instructions would be so high
<arigato>
also, the highest "getfield" is barely above the highest "setfield"
<cfbolz>
arigato: how did you count?
<cfbolz>
arigato: doesn't the virtual forcing code use blackhole too?
<arigato>
about 2 million each: getfield_vable_i, setarrayitem_vable_r, setfield_vable_i, getfield_vable_r
<arigato>
no, logic in virtualizable.py calls the same methods on the 'cpu', but that's not how I count
<arigato>
I count the dispatches to "def handler()" inside blackhole.py
<cfbolz>
Ok
<arigato>
basically I'm interested in both the overhead of some instructions, and the whole overhead of running the blackhole which is an interpreter
<cfbolz>
Ah, it's get/set on vable. That I can believe somewhat. Reads and writes on the frames come in pairs
Ubuntu-BR has quit [Ping timeout: 240 seconds]
<arigato>
with the prevalence of "*_return", we run on average 1.4 instructions before running a return
<cfbolz>
Eh
amaury_ has quit [Quit: Konversation terminated!]
amaury_ has joined #pypy
amaury_ is now known as amaury
<kenaan>
Raemi stmgc[c8-reshare-pages] e08f018e54b8 /c8/: understand and document what this branch is doing A try to write down the logic of the page reshar...
Ubuntu-BR has joined #pypy
amaury has quit [Ping timeout: 260 seconds]
rokujyouhitoma has joined #pypy
arigato has quit [Quit: Leaving]
rokujyouhitoma has quit [Ping timeout: 240 seconds]
yuyichao has joined #pypy
<fijal>
cfbolz, arigato: half of the blackhole time (last time I checked) was rebuilding stuff
ELFrederich has joined #pypy
<ELFrederich>
is it a good idea to subclass int? I'm wrapping a C library with cffi which passes around unsigned integers as "handles" to its objects. I was thinking to create a type just for clarity when in a REPL you'd see it as a <handle id=123> rather than just a plain int 123.