cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | "nothing compares to the timeshifter, my personal polar expedition in software" - pedronis
<kenaan_>
antocuni cpyext-avoid-roundtrip 87f91d94662e /: (arigo, antocuni): start a branch where to try to avoid RPython->C->RPython->C roundtrips when we...
<kenaan_>
antocuni cpyext-avoid-roundtrip 730786d85e6f /pypy/module/cpyext/: (antocuni, arigo, ronan): rewrite Py_DecRef in C
ronan has quit [Read error: No route to host]
arigato has joined #pypy
ronan has joined #pypy
antocuni has quit [Ping timeout: 248 seconds]
<kenaan_>
arigo cpyext-avoid-roundtrip 6046c5c9f4c2 /pypy/module/cpyext/: (antocuni, arigo, ronan around): same with Py_IncRef()
forgottenone has quit [Quit: Konversation terminated!]
<kenaan_>
arigo cpyext-avoid-roundtrip 10f2a25eb826 /pypy/module/cpyext/: (antocuni, ronan, arigo) Work work work until _Py_Dealloc() can be written in C
rokujyouhitoma has quit [Ping timeout: 240 seconds]
ronan has joined #pypy
<kenaan_>
arigo cpyext-avoid-roundtrip 804bbd063a8a /pypy/module/cpyext/: (ronan, antocuni, arigo) Small change to prepare the subtype_dealloc() attack
rokujyouhitoma has quit [Ping timeout: 264 seconds]
forgottenone has quit [Quit: Konversation terminated!]
kipras`away is now known as kipras
forgottenone has joined #pypy
lesshaste has quit [Quit: Leaving]
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 240 seconds]
raynold has quit [Quit: Connection closed for inactivity]
lritter has quit [Quit: Leaving]
rokujyouhitoma has joined #pypy
forgottenone has quit [Quit: Konversation terminated!]
rokujyouhitoma has quit [Ping timeout: 246 seconds]
antocuni has joined #pypy
<kenaan_>
antocuni cpyext-jit b6ca94977ea4 /pypy/doc/whatsnew-head.rst: document and close the branch
<kenaan_>
antocuni default 9f9989a1ffe6 /pypy/: merge the cpyext-jit; this makes cpyext calls up to 7x faster in very simple cases, see whatsnew for details
rokujyouhitoma has joined #pypy
<xorAxAx>
\o/ is there a sprint going on?
rokujyouhitoma has quit [Ping timeout: 240 seconds]
<xorAxAx>
antocuni:
tormoz has quit [Remote host closed the connection]
<traverseda_lapto>
Not sure if this is the right place for help with cffi, but I would like to expose a cpp class to python, inherent from it, and expose the modified class back to the cpp app. Is that reasonable? I don't really understand how cffi deals with objects.
<kenaan_>
rlamy refactor-slots d6069a5b0d69 /pypy/module/cpyext/typeobject.py: Split off the builtin case from update_all_slots()
<kenaan_>
mattip py3.5 73f012309b3c /pypy/module/posix/interp_posix.py: win32 can run without a console, return None in this case (amaury advising)
<kenaan_>
mattip py3.5 6867bc3ac3d1 /: pypy-cw.exe -> pypy-c.exe, cw is the non-console exe (amaury)
<amaury>
traverseda_lapto: cffi has not support for C++
<amaury>
what you describe can be written with cffi, but with considerable boilerplate code
<traverseda_lapto>
amaury, cool, that's more or less what I thought
<njs>
cppyy might work better, but in general this sounds like a pretty complicated trick that you might be better off finding a way to avoid
tormoz has quit [Remote host closed the connection]
<traverseda_lapto>
njs, that would be reasonable. It's a toy anyway, so I might just not do it...
<traverseda_lapto>
I want to write eaglemode widgets in python, and allow for live-editing of widgets and general introspection, but it's... not a high priority.
tormoz has joined #pypy
tormoz has quit [Remote host closed the connection]
mattip has joined #pypy
tormoz has joined #pypy
<mattip>
amaury: thanks for the pypy3.5 hints, may I pick your brain about how slow Popen via CreateProcess is on windows?
<amaury>
compared to CPython?
<mattip>
amaury: the win32 buildbot take s6 hours to run own rpython tests, it seems Popen is to blame for most of that
<amaury>
how much RAM is available to the buildbot?
<mattip>
lots
rokujyouhitoma has joined #pypy
<mattip>
I benchmarked a single test on the win32-slow-tests branch, on my machine
traverseda has joined #pypy
<mattip>
running the test_int_mul_ovf rpython\translation\backend test,
<mattip>
linux takes 1 sec to compile, 26 ms to run all the test cases,
<mattip>
windows takes 8 sec to compile, 3800 msec to run the same tests
rokujyouhitoma has quit [Ping timeout: 240 seconds]
<mattip>
same machine, booted into the OS (not a vm)
<mattip>
the test runs an exe with jitted code to multiply two integers and catch overflow, checks stdout
<mattip>
many times
<mattip>
searching a bit aboout CreateProcess and slow, it seems by design windows does much more than a fork,
<mattip>
it runs checks on the exe before starting up, antivirus and allows OS hooks
<amaury>
ah, could be the reason indeed
<njs>
mattip: it won't help with the compilation part, but can you restructure the test harness so it only spawns the exe once and then it runs lots of tests?
<amaury>
yes, the .exe needs to be read from disk, then antivirus can trigger
<amaury>
njs: I'm sure some of the tests do it this way
<amaury>
(but only a few)
<mattip>
ok, that's one idea. Any more? would it be reasonable to use os.system which avoids CreateProcess?
<njs>
mattip: ...how can os.system avoid CreateProcess? AFAIK CreateProcess is the fundamental primitive for process spawning on Windows
<njs>
I guess you could dlopen clang and compile things that way :-)
<mattip>
heh
<njs>
(sorry I mean LoadLibrary)
<mattip>
os.system calls "_wsytem" which opens a cmd and passes the string
<mattip>
s/compilation/translation/
<mattip>
what takes time at that stage AFAICT is not the compilation itself but the pre-translation stage where it does ~40 platofrm checks
<mattip>
each is a compile-run-check cycle
<mattip>
anyhow, just wondering if the wider "windows" world ever ran into this and found a work=around close to "fork"
<mattip>
closer to
Garen has quit [Read error: Connection reset by peer]
Garen has joined #pypy
<njs>
8 seconds versus 1 seconds to compile seems large for spawn overhead -- is it possible to turn down optimizations or something?
<amaury>
LoadLibrary won't help, if the antivirus is the issue
<mattip>
maybe I should look into caching results for platform tests in translation startup
<njs>
amaury: the idea / joke would be that you LoadLibrary once and then invoke it repeatedly as a function call intsead of spawning a full subprocess
<njs>
amaury: I doubt that this is practically useful though
<mattip>
njs: it might, but switching to clang would cause all kinds of other pain