cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end ) | use cffi for calling C | if a pep adds a mere 25-30 [C-API] functions or so, it's a drop in the ocean (cough) - Armin
jcea has quit [Remote host closed the connection]
adamholmberg has quit [Remote host closed the connection]
<kenaan>
antocuni dummy-importlib c37f7da0a5ee /pypy/module/_dummy_importlib/: make it possible to use __import__ directly, and use a dummy package to run tests instead of trying to i...
<kenaan>
antocuni dummy-importlib 98132b9252a9 /pypy/module/_dummy_importlib/: add enough logic to import packages and modules inside packages
<kenaan>
antocuni dummy-importlib bc01efbafe27 /pypy/module/_dummy_importlib/: import submodules with from ... import
<kenaan>
antocuni dummy-importlib2 8535b9830858 /: try again with another approach
<kenaan>
antocuni dummy-importlib2 ad9feca07a5b /pypy/: start a branch in which to introduce a _dummy_importlib module: currently, all applevel tests use _froz...
<kenaan>
antocuni dummy-importlib2 fdb244352422 /pypy/module/_dummy_importlib/interp_import.py: WIP: remove this file
<kenaan>
antocuni dummy-importlib2 e566fbc85941 /pypy/module/_dummy_importlib/interp_import.py: copy the content of pypy/module/imp/importing.py from default (rev 2e8c4536e416)
<kenaan>
antocuni dummy-importlib2 37239125314a /pypy/module/_dummy_importlib/: tweak the code until it works: remove the references to lonepycfiles and make MODULES_THAT_ALWAYS_SHADO...
<kenaan>
antocuni dummy-importlib2 3c435619e333 /pypy/module/_dummy_importlib/: add a nice error message in case something goes wrong. Add tests to check that we are actually using th...
<kenaan>
antocuni dummy-importlib 348abe425ff2 /: close dead branch
<antocuni>
ok, so I spent the last couple of days trying to get rid of importlib/_frozen_importlib during tests because the profiles showed that we spent too much time there (in particular in cpyext tests)
<antocuni>
... and now, the cpyext tests are as slow as before :(
<antocuni>
ah, it seems that my branch helps a little with test performance: on default, the 'pytest' step takes ~48 minutes on the own-linux-x86-64 builder
<antocuni>
on py3.6, 1h7min
<antocuni>
on dummy-importlib2, 55 minutes
<antocuni>
although there are failures here and there, so the final timing might be different
<__pv>
just noting it here in case it was not intentional
Rhy0lite has joined #pypy
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 260 seconds]
inhahe has quit []
marky1991 has joined #pypy
xcm has quit [Read error: Connection reset by peer]
xcm has joined #pypy
adamholmberg has joined #pypy
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
dddddd has joined #pypy
dmalcolm has quit [Ping timeout: 268 seconds]
dmalcolm has joined #pypy
jcea has quit [Quit: jcea]
dmalcolm has quit [Remote host closed the connection]
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
<antocuni>
one update on my quest to understand why cpyext tests are so slow on py3.6: it seems a huge factor is "from pickle import Picker" inside cpyext.moduledef.startup: it takes ~2s on default and ~30s on 3.6 😱
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
Ai9zO5AP has joined #pypy
<michelp>
is this an appropriate channel to ask cffi questions?
<simpson>
Yes.
<kenaan>
rlamy py3.6 67ecbe59924d /pypy/module/pypyjit/test_pypy_c/: Fix some pypyjit tests
<michelp>
it uses cffi extensively, I'm working on supporting a feature in GraphBLAS called "User Defined Types", which allows users to define a new matrix element type, and new semiring operations for that type. I'm using numba to jit compile the functions that become the semiring operations, that works well but is pretty low level for my users, I'm hoping to come up with a higher level approach
<michelp>
there's two definitions at the top, one is the "Record" style from the numba docs, and the other is a C string literal that is cdef'ed. What I'd like is to be able to call ffi.new() and construct from the Record object, but I can't seem to do that, I have to cdef the string and then call ffi.new with the type as a string. is there a way to use BF_tuple instead of the string to define the struct that ffi.new creates?
<michelp>
this might cross over more as a numba question, so I'm fine going off asking somewhere else :)
<ronan>
antocuni: maybe the main issue isn't importlib itself, but the fact that there's a big import cycle that makes up half the stdlib
<michelp>
my long goal is for my users to be able to define a new type and semiring operations as a single class, that programmatically constructs the underlying type Record and jit compiled functions
Rhy0lite has quit [Quit: Leaving]
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
mattip_ has joined #pypy
<mattip_>
__pv: it was a problem with the new buildbot, should be fixed in the next nightly
mattip_ has quit [Remote host closed the connection]
Dejan has quit [Quit: Leaving]
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
jvesely has quit [Quit: jvesely]
marky1991 has quit [Remote host closed the connection]
marky1991 has quit [Remote host closed the connection]
adamholmberg has quit [Remote host closed the connection]
<antocuni>
ronan: yes, that's what I am concluding
<antocuni>
importlib seems to be a bit slower than the default's importing.py, but the actual problem is that in py3 we import many more modules
<antocuni>
I am thinking of just adding a --cpyext-fast pytest option which disables a bunch of things (like support for pickling and LeakChecker, which also force the importing of many modules)
<antocuni>
it would not make the nightly tests any faster, but it would probably be very useful for local developmen