antocuni changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | "PyPy: the Gradual Reduction of Magic (tm)"
<arigato>
seems the .pyc files are written in text mode
<cfbolz>
ouch :-(
<arigato>
of course the import logic tries to write them in binary mode, so I suspect that our implementation of either os.open() or _io.FileIO(fd, 'wb') is broken
<arigato>
indeed, just fd = os.open(); os.write(fd) writes stuff in text mode
<arigato>
uh what, it's the same on pypy-c.exe?
<arigato>
ok it's even the same in CPython
<arigato>
seems I have no clue what os.open() is supposed to do on windows
<arigato>
right, there is os.O_BINARY that you have to give
<arigato>
and which importlib doesn't give??
<arigato>
ok, wat
<arigato>
it works because the code does os.open() forgetting O_BINARY, but then io.FileIO(fd, 'wb') somehow adds the binary again in CPython 3.x
<arigato>
that's the part missing in pypy3-c.exe
* arigato
tries on cpython 2 and pypy 2
<arigato>
no, on cpython 2 and pypy 2, the same logic really writes in text mode
antocuni has joined #pypy
<arigato>
right, the constructor of _io.FileIO(fd) whacks at the fd to turn it binary
tayfun26 has quit [Remote host closed the connection]
<kenaan>
arigo py3.5 64afb0785729 /pypy/module/_io/interp_fileio.py: Windows: missing _setmode(O_BINARY) in FileIO
<arigato>
so, running a translation up to --rtype is not enough to see a memory difference, but running it completely shows a difference (5.0 vs 6.0 GB)
<arigato>
I have no clue how this could be related to mmap-for-arenas so now I'm trying to build other versions
mattip_away has quit [Remote host closed the connection]
<fijal>
ok, so we need to have a few iterations over that, wait for armin (or me) to fix pip on windows and then kick buildbots?
<fijal>
maybe we can release before christmas
adamholmberg has joined #pypy
amaury has quit [Ping timeout: 240 seconds]
amaury has joined #pypy
<antocuni_>
who? There are places which catch VmprofPlatformUnsupported, but I cannot find any place which actually raises it
<antocuni_>
s/who/uh
amaury has quit [Quit: Konversation terminated!]
amaury_ has joined #pypy
amaury_ has quit [Client Quit]
amaury__ has joined #pypy
amaury__ has quit [Quit: Konversation terminated!]
<kenaan>
antocuni fix-vmprof-stacklet-switch-2 df87d2ec0d1b /rpython/rlib/rvmprof/cintf.py: move the ECI stuff into a proper function: this way, it will be easier to ensure that the E...
<kenaan>
antocuni fix-vmprof-stacklet-switch-2 afe32451952d /rpython/rlib/rvmprof/: Switch to a saner way to handle vmprof on unsupported platforms: - currently, we always ...
<antocuni_>
I hope that these two commits are enough to fix translation
<nanonyme>
I have DLL's with interesting installation paths that I'd want to load with full filename
mat^2 has joined #pypy
<nanonyme>
I already know this is impossible through ctypes public API but was interested whether CFFI can do it
<nanonyme>
It looked a lot like I might be disappointed based on the CFFI source code but it was fairly slow to read so I thought to ask to verify
<LarstiQ>
can you expand on what that means?
<kenaan>
antocuni fix-vmprof-stacklet-switch-2 40f2b92848b2 /: close this branch again
<kenaan>
antocuni default e742e3594267 /rpython/rlib/rvmprof/: merge again fix-vmprof-stacklet-switch-2: this should fix translation on platforms where vmprof is not supported...
raynold has joined #pypy
adamholmberg has quit [Remote host closed the connection]
<nanonyme>
Loading this is only ever possible with LoadLibraryW
<nanonyme>
This may sound like a completely stupid use case but our product supports installing into legal wide char path and I need to load DLL's from installation directory
<nanonyme>
(I already noticed subprocess under Python2 simply will never work properly for this)
<nanonyme>
(under CPython anyway, maybe PyPy doesn't have the same deficiency)
mattip has joined #pypy
dcrosta has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<nanonyme>
Neat! Looks like CFFI in fact handles this right and the UTF-8 encoding is just for outputting
<nanonyme>
So all of the new CFFI bindings I have written already support wide char paths, it's just the legacy ctypes ones that don't
<LarstiQ>
another reason not to use ctypes \o/
<nanonyme>
Yes
Taggnostr has quit [Remote host closed the connection]
<nanonyme>
That is even if subprocess called wide char API's which I think it doesn't
Taggnostr has joined #pypy
<nanonyme>
(you can probably imagine what happens in that function when args contains unicode objects which are impossible to be implicitly encoded)
<nanonyme>
Not telling you to fix it, this is just something I noticed earlier where apparently all Python2 implementations are broken on Windows
<LarstiQ>
is there a bug for that?
<nanonyme>
I doubt it. Most sane people don't even hit this. We intentionally put silly characters into installation path in CI environment just to make sure the product breaks if it uses ANSI API's
<LarstiQ>
nanonyme: time to file one then :)
<nanonyme>
LarstiQ, I don't believe CPython devs will fix it
<LarstiQ>
nanonyme: that's not the only reason to file a bug though
<nanonyme>
I mean, since this problem doesn't concern Python3
<LarstiQ>
I don't know if this qualifies for deviating from cpython code, but it might get fixed in pypy
<nanonyme>
It's very hard to get that category of things fixed when it requires large changes to standard library
<LarstiQ>
nanonyme: pypy devs care about python2
<nanonyme>
I can file one, sure
<nanonyme>
(incidentally subprocess32 would probably fix this too but no one bothered making it actually work on Windows)
<nanonyme>
Reading your source code it might well be less severe to fix for PyPy
<nanonyme>
On CPython it requires rewriting the C implementation of actually invoking process launching code on Windows somewhat but looks like yours uses CFFI and we already covered it does a sane thing
<arigato>
indeed, this is a similar file but with CreateProcessW instead of CreateProcessA
<nanonyme>
You have to go through quite a lot of loops to ensure unicode if you want to actually fix it and use CreateProcessW in both
<nanonyme>
It's a bit iffy either way: if you use W API and don't ensure unicode to both args and environment, it breaks. If you use A API and don't ensure bytes to both args and environment, it breaks
dcrosta has joined #pypy
<nanonyme>
There's no possible way of passing certain legit characters through the A API though regardless of which encoding you try to use
<nanonyme>
Hm, correction, looks like also invoking another level of crazy through using MBCS might be possible for at least larger subset of things passable through wide char API's
<nanonyme>
Sounds like I'll try that at work and file the bug at some point of using wide char API's if you're actually convinced you want to care
<nanonyme>
(IOW I'm playing around with changing default encoding from ascii to mbcs on Windows)