cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | "nothing compares to the timeshifter, my personal polar expedition in software" - pedronis
<ninjaaron>
I guess the answer for something that does this much unavoidable work is just to use cffi or something.
<ninjaaron>
hm... maybe if I just used one list at each level iteration and kept swaping out values.
<ninjaaron>
wouldn't have to keep allocating new tuples...
marr has joined #pypy
<arigato>
ninjaaron: unsure what you use case is, but itertools.product() should be as fast as it can get
<arigato>
if we knew what you are really trying to do with the result of itertools.product(), we might offer alternative ways
<ninjaaron>
arigato: OK. The pypy website mentions something about how using itertools is normally a bad idea for performance, but I don't know if there a way to do this cartesian product stuff that isn't kind of slow.
<ninjaaron>
arigato: Different stuff. I use it for a few things.
<ninjaaron>
I
<arigato>
right, but what the website means is that, if you look for a drop-in replacement of itertools, then no, there isn't one; but if you think about the problem in terms of regular for loops instead of iterators, then it can be faster
<ninjaaron>
arigato: I see.
<arigato>
for example, use three nested for loops instead of itertools.product() returning 3-tuples
<ninjaaron>
I'm trying to think of a way to do it with regular loops that still works on an arbitrary number iterables.
<arigato>
ah, that's harder, agreed
<ninjaaron>
I'm sure there is a way to speed it up from what I'm doing, but I haven't figured it out yet.
<ninjaaron>
but itertools.product is fine.
<ninjaaron>
part of it is just curiosity about the algorithms that could be used to do this.
<ninjaaron>
Its a fun one to play with.
<arigato>
:-)
Guest36286 has quit [Quit: Konversation terminated!]
<antocuni>
arigato: hi
<antocuni>
I have a couple of questions/issues about our branch
<pdox>
ninjaaron: easy
<arigato>
for example, if you use itertools.product() on 'n' lists that are always of length 2, you could replace it with a single "for i in range(2**n):"
<arigato>
you'd use bit checking in the loop
<simpson>
arigato: I used precisely that technique to win a contest years ago, to produce a fast subset-sum algorithm.
<ninjaaron>
I made the mistake of printing this set.
<simpson>
Big set? Happens.
<ninjaaron>
pdox: I didn't check the exact list produced because it's possible that they are in different orders, but the set of the two outputs is equal.
<ninjaaron>
I didn't do the conversion of the elements in the starting iterables into lists, though. I need to do that.
<ninjaaron>
yeah, this is what I was thinking about. Just use one list for everything and copy it to a new tuple before spitting it out.
<pdox>
although using yield is probably slower than forming and returning a list
<pdox>
not sure if that's true in pypy
<simpson>
It might still be faster to just generate indices. What's the algorithm on top of this?
cstratak has quit [Quit: Leaving]
cstratak has joined #pypy
jcea has joined #pypy
<kenaan>
antocuni cpyext-avoid-roundtrip 718a91452806 /pypy/module/cpyext/tupleobject.py: add a comment about a potential issue
<kenaan>
antocuni cpyext-avoid-roundtrip 0cd1a1fbbad9 /pypy/module/cpyext/: _Py_NewReference did not initialize pypy_link: as such, reused tuples from the freelist could con...
<kenaan>
antocuni cpyext-refactor-methodobject a5eb32fe2228 /pypy/module/cpyext/: merge cpyext-avoid-roundtrip inside this
antocuni has quit [Ping timeout: 255 seconds]
<mattip>
hi
<cfbolz>
hey matti!
<mattip>
cfbolz: hi
ronan has joined #pypy
raynold has joined #pypy
ninjaaron has left #pypy ["WeeChat 1.4"]
Rhy0lite has joined #pypy
Rhy0lite has quit [Ping timeout: 246 seconds]
Rhy0lite has joined #pypy
<mattip>
it would be nice to avoid the own-rpython tests (1hr*4workers on linux64, 6 hr*2workers on win32) if rpython has not changed
<mattip>
but I do not see a buildbot hook for getting last-test-run-changeset
<mattip>
there is an "onlyIfChanged" kwarg, maybe that can help?
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
drolando has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
drolando has joined #pypy
gclawes has quit [Remote host closed the connection]
gclawes has joined #pypy
drolando has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<Cheery>
"STM was a research project that proved that the idea is possible. However, the amount of user effort that is required to make programs run in a parallelizable way is significant, and we never managed to develop tools that would help in doing so."
drolando has joined #pypy
<Cheery>
there's a guy who asked about STM and lever today.. And after linking that post to him I wonder.
<Cheery>
I actually don't know much about the details here.
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
<mjacob>
pjenvey: what was the idea behind 25252fff53de0?
kipras`away is now known as kipras
exarkun has quit [Ping timeout: 248 seconds]
exarkun has joined #pypy
<pjenvey>
mjacob: at some point zipimport.c did fsencode/decoding along those lines. looks like it's changed? all I see is __init__ at a quick glance now
<mjacob>
pjenvey: i think that the filenames in zip files are encoded in ascii, cp437 or utf-8
<mjacob>
pjenvey: not sure where the fs encoding comes into play, except for the filename of the zip file itself of course
ronan has joined #pypy
<pjenvey>
sounds good to me, I just copied it from cpython
adamholmberg has quit [Remote host closed the connection]
<mattip>
something very strange in matplotlib when creating html documentation,
<mjacob>
pjenvey: it seems like cpython does it completely differently - storing unicode objects instead of bytestrings in a dict
<mattip>
there are parsing errors as if the default encoding is wrong (on pypy5.9
<mattip>
pypy2.7-v5.9)
Guest36286 has joined #pypy
Rhy0lite has quit [Quit: Leaving]
tbodt has quit [Read error: Connection reset by peer]
tbodt has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
Guest36286 has quit [Ping timeout: 255 seconds]
asmeurer_ has joined #pypy
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 248 seconds]
Guest36286 has joined #pypy
exarkun has quit [Ping timeout: 240 seconds]
exarkun has joined #pypy
asmeurer_ has quit [Quit: asmeurer_]
kolko has joined #pypy
Guest36286 has quit [Ping timeout: 248 seconds]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 240 seconds]
asmeurer_ has joined #pypy
_habnabit has quit [Ping timeout: 240 seconds]
forgottenone has quit [Quit: Konversation terminated!]
marky1991 has quit [Remote host closed the connection]
exarkun has quit [Ping timeout: 260 seconds]
exarkun has joined #pypy
jcea1 has joined #pypy
jcea has quit [Read error: Connection reset by peer]
jcea1 is now known as jcea
<kenaan>
antocuni cpyext-refactor-methodobject 17fd1d984f1d /pypy/module/cpyext/: write a specialized function to convert args_w into a py_tuple, which sets directly c_ob_it...
<kenaan>
antocuni cpyext-refactor-methodobject 371b5a6f2316 /pypy/module/cpyext/__init__.py: fix test_arraymodule:test_pickle by teaching pickle about all the W_PyCFunctionObject variants