cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | the secret reason for us trying to get PyPy users: to test the JIT well enough that we're somewhat confident about it
<kenaan>
wlav cppyy-packaging 0eee6ab380c7 /: merge default into branch
<kenaan>
wlav cppyy-packaging 4600e4a5a904 /pypy/module/_cppyy/: do not pull in the backend until the very last moment to prevent errors when importing _cppyy for doc purposes
tbodt has joined #pypy
<idnar>
so this works: `docker run --rm -ti pypy:2-6.0.0 pip install pillow`
<idnar>
but this does not: `docker run --rm -ti pypy:2-6.0.0-slim sh -c 'apt update && apt -y install zlib1g-dev strace && pip install pillow'`
<kenaan>
wlav default a4c2916c877f /pypy/module/_cppyy/: Merge of cppyy-packaging: pulls in fully deferred loading for the backend (needed for pydoc tests in p3.5 branch), s...
<kenaan>
wlav cppyy-packaging 49f83bfa0c9e /pypy/module/_cppyy/__init__.py: fix typo in comment
energizer has quit [Disconnected by services]
energizer has joined #pypy
energizer has quit [Ping timeout: 248 seconds]
tbodt has quit [Ping timeout: 276 seconds]
wleslie has joined #pypy
<nimaje>
tsutsumi: but what if someone has gcc installed, but wants to use cc (for example when cc is clang, like on freebsd)
energizer has joined #pypy
energizer has quit [Remote host closed the connection]
<RoseWine>
arigato: Under CPython 3.5.5 (and setting the multiprocessing executable to PyPy3 6.0) it gives a new error "ImportError: cannot import name 'sha512'"
<RoseWine>
I can see that CPython 3.6.5 -> CPython 3.5.5 fails too
<RoseWine>
I suppose this is probably a wontfix? Do you have any idea how to workaround?
redj_ is now known as redj
slacky has quit [Quit: Leaving]
<lesshaste>
sorry to be dim but what does PyPy3 -> CPython mean?
<lesshaste>
RoseWine,
<lesshaste>
RoseWine, I have python 3.5. Could you paste the code I should test?
<RoseWine>
I'm using multiprocessing module to call PyPy code from CPython
<lesshaste>
ImportError: No module named multiprocessing.semaphore_tracker
<RoseWine>
There are various errors, depending on what versions of the interpreters you use
<lesshaste>
ok.. I will give in :)
<RoseWine>
At first I thought it was a problem with sys.path using CPython libraries for PyPy
<RoseWine>
But I manually set sys.path for PyPy and it still fails
jcea has joined #pypy
jcea has quit [Quit: jcea]
jcea has joined #pypy
<arigato>
RoseWine: I suspect it is 'multiprocessing' which is written in such a way that it assumes that it's launching the same version of Python
<arigato>
it's bad, but little we can do to fix
<arigato>
I would guess the proper solution is not to use 'multiprocessing', but instead something else not built into the standard library
<arigato>
you might even have a chance if you took 'multiprocessing' from Python 3.5, and repackaged it under a different name (and fix all imports inside)
<Rotonen>
per default multiprocessing pickles globals and arguments at the process border
<Rotonen>
i suppose the pickles are not compatible at least?
<arigato>
the pickles should be compatible. See the tracebacks: it's more that multiprocessing from Python 3.6 sends some code to the subprocess, and that code assumes 3.6 too
<lesshaste>
the pickling situation with python is just really annoying. Just saying...
<RoseWine>
What do you mean "not compatible"? Because I checked the version and they use the same (3)
<Rotonen>
you can probably hack around that by just running a pypy subprocess in the function you wrap into the pool?
<RoseWine>
I think passing json might work better for my purposes, but I guess I'd have to build from scratch
<RoseWine>
Rotonen: I'm not sure what you mean. Because any multiprocessing that goes from CPython to PyPy seems to fail
<Rotonen>
an explicit subprocess.Popen()
solarjoe4 has quit [Quit: Leaving]
<RoseWine>
Yeah, I'd have to build the data passing from scratch though
<Rotonen>
also you'll need fairly long running processing as otherwise the overhead will kill any benefits
<RoseWine>
At the rate PyPy is going, the C support will run everything I need in pure PyPy eventually
inad924 has quit [Ping timeout: 240 seconds]
RoseWine has quit [Quit: Page closed]
antocuni has joined #pypy
<exarkun>
/part/part
exarkun has left #pypy [#pypy]
lazka has joined #pypy
<arigato>
RoseWine (for the logs): I suspect 'multiprocessing' to do something like changing PYTHONPATH before starting the subprocess
<arigato>
so basically it's really not working at all
<antocuni>
an alternative to multiprocessing is to use execnet, probably
<arigato>
you can run CPython 3.5 and say "please start pypy3", but this pypy3 will explode because it is run with PYTHONPATH pointing to the stdlib of CPython 3.5
<arigato>
so, well, I think that we should document 'multiprocessing' to be broken enough in 3.x that it can't ever start a different interpreter
<antocuni>
arigato: btw, did you have a chance to look at the question I asked 3-4 days ago about the GC? :)
dmarasca has joined #pypy
lazka has quit [Quit: Leaving]
Rhy0lite has joined #pypy
dmarasca has quit [Ping timeout: 256 seconds]
marky1991 has joined #pypy
lazka has joined #pypy
RoseWine has joined #pypy
marky1991 has quit [Ping timeout: 240 seconds]
<RoseWine>
arigato: Thanks for the update. I did notice something weird with sys.path/PYTHONPATH
<RoseWine>
but surely it's not the whole issue since if I do something like if __name__ == '__mp_main__': sys.path = ...
<RoseWine>
it still fails
<RoseWine>
I'll be gone but I'll check logs again, cheers
RoseWine has quit [Ping timeout: 260 seconds]
<tsutsumi>
nimaje: my PR only adds gcc if it can't find it on your filesystem, so it will support it, although not offer the option
<tsutsumi>
my original pr had a more complex way of discovering it that would have also detected whether cc was actually clang or gcc but Armin suggested I go with a simpler route.
dddddd has joined #pypy
<tsutsumi>
CPython has a more robust way of dealing with this, where it uses the c compiler that was used to build it. I think Pypy doesn't have a similar makefile, so people have been adding a different function to populate those values
<tsutsumi>
(CPython puts the original makefile in a location relative to distutils so it can be parsed and variables can be extracted for builds)