cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | "the modern world where network packets and compiler optimizations are effectively hostile"
rokujyouhitoma has quit [Ping timeout: 240 seconds]
vkirilichev has quit [Remote host closed the connection]
vkirilichev has joined #pypy
vkirilichev has quit [Remote host closed the connection]
ronan has joined #pypy
vkirilichev has joined #pypy
<ronan>
mjacob: my assert-rewrite would solve that issue, but requires modifying a ton of tests
<ronan>
mjacob: the hardest problem for updating pytest is asserts in pypy's app-level tests, we have code that hacks pytest's assert reinterpretation to give us app-level assert reinterpretation
ronan has quit [Ping timeout: 260 seconds]
wleslie has joined #pypy
ronan has joined #pypy
arigato has quit [Quit: Leaving]
marr has joined #pypy
antocuni has joined #pypy
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 240 seconds]
raynold has quit [Quit: Connection closed for inactivity]
<kenaan>
Dodan Enable_PGO_for_clang 10da3d6507ca /rpython/translator/c/genc.py: Enable PGO for CLang
<kenaan>
cfbolz default 75bf1a95154b /rpython/translator/c/genc.py: Merged in Dodan/pgo_clang_support/Enable_PGO_for_clang (pull request #554) Enable PGO for CLang (there's code in...
jcea has joined #pypy
wleslie has quit [Quit: ~~~ Crash in JIT!]
ronan has quit [Ping timeout: 260 seconds]
Ryanar has joined #pypy
Ryanar has quit [Client Quit]
antocuni has quit [Ping timeout: 240 seconds]
<danieljabailey>
I'm trying to translate on ARM, but I'm hitting this error: rpython/translator/c/src/stacklet/slp_platformselect.h:20:2: error: #error "Unsupported platform!"
<danieljabailey>
Is there some unsupported module I need to disable to be able to translate on ARM?
<LarstiQ>
"A Case of Computational Thinking: The Subtle Effect of
<LarstiQ>
Hidden Dependencies on the User Experience of Version
<LarstiQ>
Control"
<fijal>
arigato: you mean switch pypy -> git?
<arigato>
yes, or cffi -> git
<fijal>
I think I'm indifferent
<LarstiQ>
plus, mercurial has the evolve extension that beats anything git has to offer in that area
<fijal>
(but there are good reasons to use github)
<realitix>
yes github is THE good reason
<realitix>
you know what I think, it would be good for pypy and cffi to be on github (more pull request, more visibility...)
<realitix>
but bad for your brains ;-)
<realitix>
at first
<antocuni>
I must say that after having used git for a while, I start to see some advantages over mercurial
<antocuni>
although my main complain is that once you merge a branch, you loose the fact that a specific commit was developed on such a branch
<realitix>
autocuni, if you do a merge, you can see the merge commit so you know the branch which was used. Personnaly, I prefer rebasing to keep a clean history
<antocuni>
yes exactly, the name of the branch remain only in the commit message; not exactly "metadata" :)
<LarstiQ>
realitix: I think what antocuni means is that if you delete the branch pointer there is no data of what the name was
<Alex_Gaynor>
antocuni: this is also true if you use bookmarks in hg (which is what I'm told you are supposed to use these days)
<antocuni>
in pypy we still use named branches :)
<fijal>
realitix: either way the showstopper is transition
<realitix>
fijal: of course :-(
<fijal>
so if you make a plan for transition we can consider it
<fijal>
but we'll *definitely* not do the transition ourselves
<realitix>
I can first take a look at cffi which should be easier to migrate but I will write notes about that transition.
rokujyouhitoma has joined #pypy
<fijal>
realitix: you're fighting more uphill battle with cffi though :)
<fijal>
since it boils down essentially to "convince armin" and that requires very convincing arguments
<realitix>
lol
<fijal>
like, cffi does not need more exposure
<antocuni>
in the case of pypy, I suppose the biggest pain will be migrating all the related integration services we have (e.g. buildbot, the commit emails, the irc bot, etc.)
<fijal>
or anything more, really
<realitix>
armin is trying git, so maybe he will reconsider :-)
<fijal>
doubt it
<fijal>
if anything he's regretting that :)
<arigato>
that's right
<realitix>
^^
vkirilichev has quit [Remote host closed the connection]
rokujyouhitoma has quit [Ping timeout: 240 seconds]
<mjacob>
ronny: it isn't up to date, but recently it broke completely
<mjacob>
ronan (logs): in which way would it require modification of tests?
<mjacob>
arigato: is there any thing i have to keep in mind when implementing async generator's a aclose() method?
<mjacob>
arigato: or did you just insert the "XXX"s to make to task smaller?
<arigato>
the "XXX" just means "not implemented so far"
<arigato>
I've gone as far as trying to write a test, but failing (py.test -A fails)
<arigato>
so feel free to do it
<mjacob>
is the test in tree?
<arigato>
no
<mjacob>
"py.test -A fails" means that it fails on CPython?
<arigato>
yes
<arigato>
so it's useless as it is
<mjacob>
does it test for the wrong thing is there a bug in CPython?
<mjacob>
+or
<mjacob>
i couldn't think of a better way to test it than catching GeneratorExit in the generator
realitix has quit [Quit: Leaving]
antocuni has quit [Ping timeout: 240 seconds]
<ronny>
mjacob: so atm there aint much i can do to help
ronan has joined #pypy
<mjacob>
ronny: ok
rmariano has quit [Ping timeout: 276 seconds]
Tiberium has joined #pypy
<ronan>
mjacob: the trick is to add an assert_() function that flowspace handles exactly like an assert statement
<ronan>
mjacob: so you need to s/assert foo/assert_(foo)/ in every piece of RPython code in every test file
<ronan>
mjacob: also, how is it 'completely broken'?
<mjacob>
ronan: i meant that until to some particular point of time, it mostly worked for me and now it doesn't even run one test because rpython/pytest.ini contains "addopts = --assert=reinterp -rf"
<ronan>
mjacob: ah, yes, we currently require pytest<3, but 2.9 works
<mjacob>
right
rmariano has joined #pypy
Ryanar has quit [Quit: Ryanar]
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 260 seconds]
vkirilichev has joined #pypy
oberstet has quit [Ping timeout: 260 seconds]
Ryanar has joined #pypy
oberstet has joined #pypy
iko has quit [Remote host closed the connection]
adamholm_ has joined #pypy
adamholmberg has quit [Ping timeout: 255 seconds]
Tiberium has quit [Remote host closed the connection]
mattip has joined #pypy
adamholm_ has quit [Remote host closed the connection]
adamholmberg has joined #pypy
<kenaan>
mjacob py3.6 a52d37ceba11 /pypy/interpreter/: Add first test for aclose() on async generator and make it pass.
vkirilichev has quit [Remote host closed the connection]
Ryanar has joined #pypy
Ryanar has quit [Client Quit]
<kenaan>
mjacob py3.6 7f4ec0d17965 /pypy/interpreter/generator.py: Add a TODO.
<kenaan>
mjacob py3.6 3de6fb333784 /pypy/interpreter/: Implement AsyncGenABase.descr_close(). Translation should work again!
rmariano has quit [Ping timeout: 255 seconds]
<njs>
mjacob: arigato: I think I mentioned this before, but in case it's useful, https://github.com/njsmith/async_generator has a pretty complete pure-python (app-level) implementation of async generators, so might be useful for inspiration, or if you want to steal my test suite
asmeurer__ has joined #pypy
<mjacob>
njs: thank you very much, i already took inspiration from one of your tests
<njs>
fijal: hey, I modify the frame of a running generator!
<njs>
(...but it's pretty terrible yes)
<fijal>
njs: well I'm glad we all pay for your hobbies?
rokujyouhitoma has quit [Ping timeout: 240 seconds]
<njs>
fijal: I guess if one wanted to make a serious point it would be that this kind of flexibility can be useful as an escape valve for extreme situations, but uh yeah I can't really be bothered
<fijal>
I'm not really against that thing being there at all tbh
<njs>
fijal: really I just want to attach some metadata to frames, and it turns out that the only place to do that is by mutating locals()
<fijal>
since there is always debuggers
asmeurer__ has quit [Quit: asmeurer__]
<fijal>
yeah.... you should not attach metadata to frames
<fijal>
you completely wreck performance on pypy this way FYI
<njs>
trio is an extremely weird case, it's really a language extension disguised as a library
<fijal>
like, pypy is unable to JIT trio kind of bad
<njs>
but anyway, this is not really here or there WRT what you're saying in the talk, it's just that that line jumped out at me :-)
<fijal>
heh yeah
asmeurer_ has joined #pypy
<njs>
fijal: trio microbenchmarks get 5x-10x faster with pypy
<fijal>
... potentially ;-)
<fijal>
I think we are just not smart enough with generators
<njs>
fijal: fundamentally the thing I want to do is sensible, but there are limitations in the language that make it annoyingly tricky
<njs>
fijal: re "no notion of binary" and spurious copies, is this something that you've found in profiles, or...?
<fijal>
in difficulty to support to start with
<fijal>
but yeah
<fijal>
lack of the idea of "this is how binary should look like" is quite abnoyng
<fijal>
annoying
<njs>
fijal: I'm curious because I've been annoyed at how hard python makes it to avoid copying buffers, but I'm also dubious that it actually matters for most realistic python programs
<fijal>
so, it definitely matters if you try to write something fast
<fijal>
e.g. networking library
<fijal>
or try to compete with C
<fijal>
I'm not 100% sure if it matters for some significant percentage of libraries
<fijal>
it matters for twisted tcp benchmark for sure
<njs>
fijal: well, networking libraries is what I'm talking about :-). But like, the common pattern for a protocol parser in python is buf = bytearray(); buf += sock.recv(16384); buf[:1000]; del buf[:1000]
<fijal>
that all does copies though no?
<njs>
fijal: so there's 2 copies there explicitly, plus O(1) extra copy for the bytearray's occasional realloc
iko has joined #pypy
<fijal>
you could have zero with recvinto
cstratak has quit [Ping timeout: 255 seconds]
<fijal>
and that does not work well on pypy because reasons
<njs>
fijal: but I don't think I've ever seen these show up in profiles. memcpy is *really* fast.
<fijal>
but then you do encode('utf8')
<fijal>
and that does extra copies
<fijal>
memcpy yes
<fijal>
but allocating memory isn't
<fijal>
so your cache suffers
<fijal>
it won't show up in memcpy though
<njs>
it's true, it does increase cache overhead a bit, and that cost is diffuse across the profile
<njs>
zero-copy solutions also add a fair amount of complexity, like now you want me to do a regex search across a linked list of buffers?
asmeurer_ has quit [Quit: asmeurer_]
<pjenvey>
fijal: how's that utf8 repr support going
<fijal>
njs: so even if you copy it to one buffer
<fijal>
half of the APIs you want to use is on something else or accepts something else
<fijal>
pjenvey: I didn't look much
<njs>
fijal: I have filed a certain number of bugs to get people to support bytearray, yes :-)
<fijal>
see?
<fijal>
njs: anyway, what do you think?
<fijal>
and then there is memoryview, array.array etc.
<njs>
memoryview avoids copies, but usually makes things slower IIRC :-)
<njs>
fijal: about the talk overall? seems accurate enough :-)
<fijal>
njs: because noone supports it :)
<fijal>
but that's exactly what I mean by "binary"
<fijal>
even if you settle on one (imperfect) binary form, it's better than the current setup