cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | the secret reason for us trying to get PyPy users: to test the JIT well enough that we're somewhat confident about it
jcea has joined #pypy
marr has quit [Ping timeout: 256 seconds]
pf_moore has quit [Quit: Connection closed for inactivity]
AndrewBC has joined #pypy
jcea has quit [Quit: jcea]
tbodt has joined #pypy
forgottenone has joined #pypy
sbauman has quit [Quit: Connection closed for inactivity]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
energizer has joined #pypy
<cfbolz> finally a branch merge of mine that didn't cause fallout ;-)
energizer has quit [Remote host closed the connection]
<arigato> :-)
<arigato> mattip: I'll review this evening
<arigato> leaving soon for the day
mattip is now known as Guest88607
Guest88607 has quit [Killed (cherryh.freenode.net (Nickname regained by services))]
mattip_ has joined #pypy
mattip has joined #pypy
<mattip> arigato: hi
<arigato> hi
<mattip> any chance you could take a look at the cpyext-subclass-setattr branch?
<mattip> it solves the issue I was having, might it cause new ones (tests pass, but ...)
<mattip> ?
tayfun26 has joined #pypy
<arigato> yes, I'll review this evening. Leaving soon for the day
<mattip> cool, thanks
mattip has quit [Remote host closed the connection]
oberstet has joined #pypy
energizer has joined #pypy
glyph has quit [Quit: End of line.]
glyph has joined #pypy
asmeurer has quit [Quit: asmeurer]
marr has joined #pypy
energizer has quit [Quit: Leaving]
Hasimir has quit [Read error: Connection reset by peer]
Hasimir has joined #pypy
pf_moore has joined #pypy
tayfun26 has quit [Read error: Connection reset by peer]
oberstet has quit [Remote host closed the connection]
dddddd has joined #pypy
marr has quit [Ping timeout: 276 seconds]
jcea has joined #pypy
jcea has quit [Remote host closed the connection]
jcea has joined #pypy
forgottenone has quit [Read error: Connection reset by peer]
mattip_ is now known as mattip
tos9 has quit [Ping timeout: 264 seconds]
tos9 has joined #pypy
lazka has joined #pypy
forgottenone has joined #pypy
lazka has quit [Ping timeout: 240 seconds]
lazka has joined #pypy
<lazka> so, tp_dictoffset doesn't work with cpyext?
<mattip> lazka: do you have a reproducable failure?
marr has joined #pypy
lazka_ has joined #pypy
<mattip> lazka_: do you have a reproducable failure?
<lazka_> sorry, my ISP is dropping random connections since a few days
lazka has quit [Ping timeout: 256 seconds]
lazka has joined #pypy
<lazka> mattip, yeah, calling tp_setattro doesn't create the instance dict, at least not at that offset
lazka_ has quit [Ping timeout: 248 seconds]
<mattip> lazka: pastebin an example?
<lazka> mattip, I can try to make a smaller reproducer, just wanted to check if it's supposed to be supported
<lazka> I can work around it I think, if needed
marky1991 has joined #pypy
<mattip> lazka: indeed, it seems we never worked out the details of how to do it
<mattip> it seems you should use PyObject_GetAttr, PyObject_SetAttr and not directly call the tp_* functions
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
tayfun26 has joined #pypy
drolando has quit [Remote host closed the connection]
<lazka> it's while chaining up in tp_setattro
drolando has joined #pypy
<lazka> so I have to use tp_ I think
<kenaan> mattip cpyext-subclass-setattr dc1bf38e9aed /pypy/module/cpyext/test/: test places that use is_cpytype, reomve noisy print statements
<lazka> some context: we just have a custom __dict__ to check if the object gets any state, so we have to keep the wrapper around in case we wrap the same C object again later
<bbot2_> Started: http://buildbot.pypy.org/builders/own-linux-x86-64/builds/6658 [mattip: force build, cpyext-subclass-setattr]
<bbot2_> Started: http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/5379 [mattip: force build, cpyext-subclass-setattr]
irclogs_io_bot has quit [Remote host closed the connection]
<mattip> lazka: sounds advanced. Can you do it at app-level, not in c-api?
<lazka> I'll just get the instance dict and check its size instead on PyPy
<mattip> +1
irclogs_io_bot has joined #pypy
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
<bbot2_> Failure: http://buildbot.pypy.org/builders/own-linux-x86-64/builds/6658 [mattip: force build, cpyext-subclass-setattr]
marr has quit [Ping timeout: 240 seconds]
lazka has quit [Read error: Connection timed out]
forgottenone has quit [Quit: Konversation terminated!]
<kenaan> mattip cpyext-subclass-setattr 268b6bc566ac /pypy/module/cpyext/test/test_arraymodule.py: reverse logic
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
<bbot2_> Success: http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/5379 [mattip: force build, cpyext-subclass-setattr]
planrich has joined #pypy
marky1991 has quit [Ping timeout: 260 seconds]
forgottenone has joined #pypy
forgottenone has quit [Read error: Connection reset by peer]
lazka has joined #pypy
marky1991 has joined #pypy
<lazka> mattip, works now :)
<lazka> some tests are still failing, but all programs I throw at it seem to work fine now
raynold has quit [Quit: Connection closed for inactivity]
<mattip> lazka: is the speed tolerable?
Rhy0lite has joined #pypy
lazka has quit [Ping timeout: 264 seconds]
lazka has joined #pypy
<__pv> https://bitbucket.org/pypy/pypy/issues/2781 --- trying to test scipy on current default and hitting this, so wondering if it's reproducible...
<lazka> mattip, startup time is 2x more, memory is 4x larger. UI interaction is fine, as it's mostly spend in C anyway
lazka has quit [Quit: Leaving]
WhatisRT has joined #pypy
<WhatisRT> hi, I'm trying to use the cffi library and am running into problems
<WhatisRT> I'm trying to link a c library which was built for x86_64, but for some reason it wants a library for i386
<WhatisRT> there are flags -arch i386 -arch x86_64 passed to cc
<simpson> Hm. And which CPU are you running on? I haven't seen this before.
<WhatisRT> some recent i5 CPU on a mac
<WhatisRT> ld: warning: ignoring file ./libfoo.dylib, file was built for x86_64 which is not the architecture being linked (i386): ./libfoo.dylib
<WhatisRT> that's the exact error message
<simpson> Oh, I don't know anything about OSX, sorry.
<WhatisRT> do you know how I could get rid of the -arch i386 flag?
<WhatisRT> if I execute the command manually without it, it works
<simpson> Nope, I don't know what's choosing both of those flags.
illume has joined #pypy
raynold has joined #pypy
<WhatisRT> ok, now I have the next issue: I've built my example code but it does not find the library that lies in the same folder
<WhatisRT> ImportError: dlopen(..../python/_example.so, 2): Library not loaded: @rpath/libfoo.dylib
<WhatisRT> I have passed extra_link_args=['-L.'] to set source so it would accept this library
<WhatisRT> putting the library in my PATH does not help
<mattip> __pv: "hitting this" - so you see the leak?
<mattip> WhatisRT: on linux you need to set LD_LIBRARY_PATH not PATH, dunno if that works on macOS too
<cfbolz> mattip: I assume they reported the bug
tayfun26 has quit [Quit: tayfun26]
<WhatisRT> mattip: thanks, I think that worked (i.e. the issue I'm having now seems unrelated to cffi)
<mattip> WhatisRT: cool
drolando has quit [Remote host closed the connection]
drolando has joined #pypy
<mattip> working on issue 2776, it seems cpython always calls ioctl(fd, op, arg) with at least 1024 bytes in arg
<mattip> according to "man ioctl", the number of bytes is actually encoded in the op, and ioctl.h says it is encoded in the "lower 14 bits of the upper 16 bits"
<mattip> do we take the pragmatic "do what cpython does" or the correct "parse the arg size from op" ? I think the pragmatic :)
<njs> mattip: pretty sure trying to parse op would be totally non-portable
<njs> mattip: I think what the man page means is "the kernel will look at the operation and then figure out how much data to read from userspace using a totally ad hoc method invented by some random driver author"
<njs> I wouldn't be surprised at all to see an ioctl with a variable-length arg
stduolc has joined #pypy
<mattip> njs: yes, pragmatic beats correct in this case
<njs> mattip: the options are literally "pragmatic but not 100% correct" or "impractical and also wrong" so I think you're ok :-)
<njs> https://lwn.net/Articles/48354/ <-- apparently the size encoded in the op is entirely advisory, and for historical reasons involving a disagreement over who was supposed to call sizeof, lots of ioctls have a size field of "4"
<njs> (because that's sizeof(sizeof(the_actual_structure)))
<njs> (aren't you glad you asked)
AndrewBC has quit [Read error: Connection reset by peer]
<__pv> mattip: yes, I see the leak, both in the reduced example in the issue and scipy testsuite (which then bombs because of it)
<mattip> __pv: ok, thanks for the report. I will try to look this week
<mattip> i wonder if valgrind can find it on a debug numpy build
illume has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
asmeurer__ has joined #pypy
illume has joined #pypy
asmeurer__ has quit [Quit: asmeurer__]
energizer has joined #pypy
tbodt has joined #pypy
asmeurer__ has joined #pypy
stduolc has quit [Quit: Connection closed for inactivity]
drolando has quit [Read error: Connection reset by peer]
drolando has joined #pypy
energizer has quit [Ping timeout: 260 seconds]
jcea has quit [Quit: jcea]
asmeurer__ has quit [Quit: asmeurer__]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
Rhy0lite has quit [Quit: Leaving]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
illume has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
WhatisRT has quit [Ping timeout: 256 seconds]
tormoz has quit [Read error: Connection reset by peer]
marr has joined #pypy
tbodt has joined #pypy
tormoz has joined #pypy
illume has joined #pypy
marky1991 has quit [Ping timeout: 276 seconds]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
marky1991 has joined #pypy
<mattip> __pv: I think I found the leak, in slotdefs.py we create a tuple object in wrap_call, wrap_init but never decref it
<mattip> it's late now, but I will try to fix it soon
tbodt has joined #pypy
<__pv> thx
<ronan> mattip: I'm not sure wrap_call is the issue, we also leak from PyObject_CallFunctionObjArgs
antocuni has joined #pypy
<mattip> ronan: take a look at wrap_call, wrap_init. Maybe _also_ PyObject_CallFunctionObjArgs
<mattip> I added a for i in range(10): gc.collect() to the end of the script, which cleared some false positives
<nanonyme> Why would calling gc.collect multiple times have different results?
<cfbolz> nanonyme: objects with finalizers can take a while to die
* mattip zzz
<ronan> mattip: ah yes, it seems that antocuni's optimisations cause the tuple to not be collected any more
<antocuni> which optimization?
<ronan> antocuni: from cpyext-fast-typecheck, I think
<ronan> the PyTuple_Object used to be created implicitly from the w_args, but now we need to manage its refcount explicitly
<antocuni> ah I see; I reuse the same logic for function calls, but probably function calls also decref a tuple
<antocuni> sorry for that :(
<antocuni> but I wonder: why the tests didn't find it?
<ronan> the tests can't check for leaks
<antocuni> I thought they do
<antocuni> I'm quite sure I have already seen some tests failing because of a refcount leak
<antocuni> oh, or maybe it was a raw_malloc() leak?
<ronan> no, leakchecking is disabled due to too many false positives
<antocuni> ah
<antocuni> good to know
<ronan> *is disabled in cpyext
illume has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
tbodt has quit [Read error: Connection reset by peer]
tbodt has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
antocuni has quit [Ping timeout: 256 seconds]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
jcea has joined #pypy
drolando has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
adamholmberg has joined #pypy
drolando has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
sbauman has joined #pypy
tbodt has joined #pypy
planrich has quit [Ping timeout: 276 seconds]
drolando has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 264 seconds]
drolando has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
tbodt has quit [Client Quit]
marky1991 has quit [Read error: Connection reset by peer]
asmeurer_ has joined #pypy
drolando has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
pf_moore has quit [Quit: Connection closed for inactivity]
drolando has joined #pypy