cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | "the modern world where network packets and compiler optimizations are effectively hostile"
bendlas has joined #pypy
gclawes has joined #pypy
marr has quit [Ping timeout: 240 seconds]
raynold has joined #pypy
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 246 seconds]
deep-book-gk has joined #pypy
deep-book-gk has left #pypy [#pypy]
jsn1993 has joined #pypy
jsn1993 has quit [Client Quit]
shunning has joined #pypy
shunning has left #pypy [#pypy]
Ryanar has joined #pypy
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 260 seconds]
walt has joined #pypy
walt is now known as Guest12516
Guest12516 has quit [Client Quit]
Ryanar has quit [Quit: Ryanar]
[0__0] has joined #pypy
[0__0] has quit [Remote host closed the connection]
[0__0] has joined #pypy
rokujyouhitoma has joined #pypy
ArneBab_ has joined #pypy
rokujyouhitoma has quit [Ping timeout: 260 seconds]
ArneBab has quit [Ping timeout: 276 seconds]
q4 has joined #pypy
_whitelogger has joined #pypy
rokujyouhitoma has joined #pypy
yuyichao has quit [Read error: Connection reset by peer]
yuyichao has joined #pypy
rokujyouhitoma has quit [Ping timeout: 260 seconds]
forgottenone has joined #pypy
q4 has quit [Quit: Textual IRC Client: www.textualapp.com]
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 260 seconds]
utek has joined #pypy
_whitelogger has joined #pypy
GK1wmSU has joined #pypy
GK1wmSU has left #pypy [#pypy]
_GK1wmSU has joined #pypy
_GK1wmSU has left #pypy [#pypy]
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 260 seconds]
TheAdversary has quit [Remote host closed the connection]
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 260 seconds]
santagada_ has joined #pypy
santagada_ is now known as santagada
vkirilichev has joined #pypy
marky1991 has quit [Ping timeout: 258 seconds]
forgottenone has quit [Quit: Konversation terminated!]
forgottenone has joined #pypy
marky1991 has joined #pypy
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 240 seconds]
forgottenone has quit [Quit: Konversation terminated!]
marky1991 has quit [Ping timeout: 260 seconds]
marr has joined #pypy
forgottenone has joined #pypy
realitix has joined #pypy
cstratak has joined #pypy
cstratak has quit [Client Quit]
cstratak has joined #pypy
raynold has quit [Quit: Connection closed for inactivity]
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 248 seconds]
forgottenone has quit [Quit: Konversation terminated!]
antocuni has joined #pypy
<kenaan> arigo cffi/cffi 5052e1026bbd /c/_cffi_backend.c: Minor fix
oberstet has joined #pypy
<kenaan> arigo gc-del-limit-growth f0335bf5364e /: Close branch, ready to merge
<kenaan> arigo default 44577e4653fa /rpython/memory/: hg merge gc-del-limit-growth Issue #2590: fix the bounds in the GC when allocating a lot of objects with finalizers
<kenaan> arigo default abb9f6f4b003 /pypy/objspace/std/: Fix for b43a6e2c0ea1: can't very reasonably use appexec() here, because it would create a single jit loop. If the ...
ctismer_afk is now known as ctismer
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 260 seconds]
antocuni has quit [Ping timeout: 258 seconds]
rokujyouhitoma has joined #pypy
forgottenone has joined #pypy
rokujyouhitoma has quit [Ping timeout: 260 seconds]
nimaje1 has joined #pypy
nimaje is now known as Guest53784
nimaje1 is now known as nimaje
Guest53784 has quit [Ping timeout: 260 seconds]
lritter has joined #pypy
forgottenone has quit [Quit: Konversation terminated!]
forgottenone has joined #pypy
forgottenone has quit [Quit: Konversation terminated!]
forgottenone has joined #pypy
rokujyouhitoma has joined #pypy
<kenaan> rlamy py3.5 46c9f0976aad /pypy/module/cpyext/test/test_cpyext.py: Un-xfail a test; delete one that doesn't make sense on pypy3
rokujyouhitoma has quit [Ping timeout: 240 seconds]
oberstet has quit [Ping timeout: 268 seconds]
oberstet has joined #pypy
bendlas has quit [Ping timeout: 240 seconds]
yuvipanda has quit [Ping timeout: 276 seconds]
* arigato merges default -> py3.5
antocuni has joined #pypy
<kenaan> arigo py3.5 2ae11d0d2965 /: hg merge default
Rhy0lite has joined #pypy
forgottenone has quit [Read error: Connection reset by peer]
forgottenone has joined #pypy
<antocuni> arigato, fijal: do you have opinions about this bug? https://bitbucket.org/pypy/pypy/issues/2593/distutils-runtime_library_dirs-is-broken
<arigato> antocuni: not really
<fijal> neither
<antocuni> one quick workaround could be to hardcode 'gcc' if we are on linux
<antocuni> another is to actually invoke 'cc --version' and check the output to see if it's gcc
<antocuni> both of them sounds suboptimal :(
marky1991 has joined #pypy
<arigato> does pypy run on any non-windows platform where 'gcc' is unlikely to work?
<antocuni> probably not
<fijal> OS X
<antocuni> ah
<fijal> (gcc defaults to clang, but it's not a given)
<nimaje> freebsd
<fijal> error: unrecognized command line option ‘-R’; did you mean ‘-R’?
<fijal> I love that error msg
<antocuni> fijal: what does sysconfig.get_config_var('CC') return on OS/X?
<antocuni> (on CPython)
<fijal> which cpython?
<fijal> 'cc'
<antocuni> ok, so probably "hardcode gcc is it's linux" should be ok?
<fijal> I don't know
<fijal> the problem with "OS construction kit" is that they end up differently
<fijal> potentially
<antocuni> yes, but I think that the only alternative is to invoke 'cc --version', and I'm not sure I like it :(
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 255 seconds]
Ryanar has joined #pypy
_main_ has joined #pypy
__main__ has quit [Ping timeout: 260 seconds]
_main_ is now known as __main__
adamholmberg has joined #pypy
raynold has joined #pypy
walt has joined #pypy
walt is now known as Guest28777
__main__ has quit [Read error: Connection reset by peer]
__main__ has joined #pypy
slackyy has joined #pypy
<kenaan> rlamy cpyext-leakchecking e4f438ccf573 /pypy/module/cpyext/test/test_cpyext.py: Add more information when the leakfinder finds an error
slackyy has quit [Read error: Connection reset by peer]
slackyy has joined #pypy
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 260 seconds]
slackyyy has joined #pypy
slackyyy has quit [Client Quit]
realitix has quit [Ping timeout: 240 seconds]
slackyy has quit [Ping timeout: 255 seconds]
irclogs_io_bot has quit [Remote host closed the connection]
yuyichao has quit [Ping timeout: 246 seconds]
yuvipanda has joined #pypy
irclogs_io_bot has joined #pypy
yuyichao has joined #pypy
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 260 seconds]
bendlas has joined #pypy
vkirilichev has quit [Remote host closed the connection]
<Guest28777> Hi pypy experts, I would like to learn how to write a pypy mixed module. Is there any suggestions? I've read the dev documentation, some code under module/micronumpy and objspace/std.
exarkun has joined #pypy
<fijal> Guest28777: so we would suggest "not to do it"
<fijal> why do you want to?
<kenaan> rlamy cpyext-leakchecking 466144c090c7 /pypy/module/cpyext/test/test_cpyext.py: Preload a few more builtin types
<kenaan> rlamy cpyext-leakchecking d4f923fa8dfa /pypy/module/cpyext/: fix refleaks in test_tupleobject
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 260 seconds]
forgottenone has quit [Quit: Konversation terminated!]
forgottenone has joined #pypy
<Guest28777> fijal: Thanks for the reply fijal! I would like to implement some data type and understand the gap between pypy+jit vs pypy builtin, actually after seeing your micronumpy post https://morepypy.blogspot.com/2009/07/pypy-numeric-experiments.html
<fijal> I can maybe explain?
<fijal> how si your data type looking?
mihneas has joined #pypy
<Guest28777> Awesome, my data type is a class wrap around python int with the ability to slice some bits of the int like: x = 7; x[0:2] = 0; assert x == 4;
<antocuni> Guest28777: you might be interested in this: https://github.com/antocuni/capnpy/blob/master/capnpy/ptr.py
<antocuni> in that code, I use bitwise operations to extract some bits from integers
<antocuni> and I confirm that the JIT handles this kind of code very well
<Guest28777> In __getitem__/__setitem__ of the class, I manipulate the bits by shifting. I look at pypy log and found that there are a lot of trace generated but got optimized away in the end. Since I'm doing a lot of these in a single iteration, I have to increase the trace_limit to avoid trace aborts
<antocuni> right, this is a well known problem
<Guest28777> antocuni: Thanks for letting me know about this. The JIT works well with my code as well, it's just the trace_abort problem. As a result, I thought implement my data type as a mixed module might help reduce the trace length. This is just my speculation, not sure if this is true though.
cstratak has quit [Ping timeout: 260 seconds]
<antocuni> yes, it might help. Although we should probably fix this trace limit problem, eventually :(
__Myst__ has joined #pypy
<__Myst__> Can you use Pillow on pypy?
<fijal> __Myst__: it should work
<__Myst__> I get a `cannot import _imaging` error
<fijal> __Myst__: you need to pip install it (with pypy)
<fijal> I suggest using a virtualenv
<__Myst__> I did, it says requirement already satisfied
<mihneas> Hello, I am trying to implement a missing feature in PyPy and I don't know how to modify the tests in the lib-python folder to make sure my added code works properly . Can someone give me a piece of advice ?
<__Myst__> fijal: it works with a pypy venv, thank you very much. may i ask why this is the case?
<fijal> __Myst__: you may ask but the answer is "python import system is incredibly complicated and I don't know"
<fijal> mihneas: you need to be a bit more specific
<Guest28777> antocuni: I guess I have to deal with the trace limit problem :(. Do you have any suggestion on implement a mixed module?
tilgovi has joined #pypy
<fijal> Guest28777: can you increase the trace limit simply?
<fijal> there are command line parameters (as well as APIs)
<Guest28777> Yes, I did
<antocuni> __Myst__: most probably, when you are not inside the virtualenv you do "pip install" and "pip" points to CPython's pip
<antocuni> __Myst__: if you do pypy -m pip install, it should work even without the virtualenv
oberstet has quit [Ping timeout: 260 seconds]
<fijal> Guest28777: didn't help?
<antocuni> fijal: the problem is that the max limit is 16384
<Guest28777> fijal: I also discovered that I have to remove an assertion to get to > 16384 trace limit
<__Myst__> antocuni: no, i do do `pypy3 -m pip`, it does tell me that the requirement is already satisfied from /usr/lib/python3.5
<mihneas> @fijal I`ve got AssertionError: DID NOT RAISE
<mihneas> AssertionError: DID NOT RAISE
<mihneas> @fijal I`ve got AssertionError: DID NOT RAISE
<fijal> antocuni: ah :)
<antocuni> fijal: no, it's a real problem I think. I got bitten by it already few times, at some point we need to think about a better strategy for long traces
<Guest28777> antocuni: You can add bigmodel option to compile pypy, and remove the assertion. I can use 1M trace limit.
<antocuni> __Myst__: then I don't know :)
<fijal> antocuni: yop
<antocuni> Guest28777: cool :)
<fijal> Guest28777: this is something to think about then
<antocuni> anyway, /me --> away
<Guest28777> and then compile like this python ../../rpython/bin/rpython --translation-jit_opencoder_model big -Ojit targetpypystandalone
<Guest28777> Then you can enjoy up to 1 billion trace length
<Guest28777> Just my dirty hack :(
Guest28777 has left #pypy [#pypy]
shunning has joined #pypy
shunning has quit [Client Quit]
adamholm_ has joined #pypy
shunning has joined #pypy
<shunning> After I increased the trace limit to around 1 million, all the trace aborts were just gone.
<shunning> I think it's just because my single iteration was too large, not because pypy doesn't like my code
slackyy has joined #pypy
antocuni has quit [Ping timeout: 255 seconds]
adamholmberg has quit [Ping timeout: 260 seconds]
rokujyouhitoma has joined #pypy
<shunning> fijal: It would be very useful if you could comment on these arguments.
<shunning> 1. Writing a mixed module might reduce the trace length (and hence warmup time).
<fijal> Guest29691: yeah, that's not gonna work....
<shunning> 2. Since the JIT already handles this data manipulation well, would I actually get performance boost if I write a mixed module?
<fijal> shunning: that's not gonna work, there is a hard limit on data representation (we can have a different pypy that uses more bits)
<fijal> you're right, but it's also a bad way to approach this
<fijal> I mean, if you really insist, it's doable
<fijal> we discourage it, but it's doable
rokujyouhitoma has quit [Ping timeout: 268 seconds]
<shunning> Thanks for the comments. What would be a good way?
adamholm_ has quit [Read error: Connection reset by peer]
adamholmberg has joined #pypy
slackyyy has joined #pypy
slackyy has quit [Ping timeout: 240 seconds]
<fijal> good way would be fixing the jit traces limit problem
<fijal> like first of all, why is it so high?
<fijal> I understand if you don't want to fix it
oberstet has joined #pypy
<shunning> I think I'm just doing a tons of them in a single loop iteration.
<shunning> I tried to separate them into different traces by creating an if, but the performance degrade for a bit
<exarkun> fijal: Hiya
shunning has quit [Quit: Page closed]
glyph has quit [Quit: End of line.]
glyph has joined #pypy
dmalcolm has quit [Read error: Connection reset by peer]
dmalcolm has joined #pypy
adamholm_ has joined #pypy
adamholmberg has quit [Ping timeout: 240 seconds]
rokujyouhitoma has joined #pypy
ronan has quit [Ping timeout: 276 seconds]
rokujyouhitoma has quit [Ping timeout: 240 seconds]
<exarkun> is the 3.5 branch expected to translate on cpython 2.7?
ronan has joined #pypy
tilgovi has quit [Ping timeout: 240 seconds]
forgottenone has quit [Quit: Konversation terminated!]
forgottenone has joined #pypy
<exarkun> hm, also fails on pypy 5.4.1
<exarkun> I suppose due to default being merged into it recently, with changes that aren't python 3 compatible (somehow)
<exarkun> 91998 specifically, perhaps
<exarkun> mixed bytes/unicode. the idiom for this case seems to be to encode with utf-8...
<cfbolz> exarkun: what's the hash of the commit?
<exarkun> 2ae11d0d2965 ?
<exarkun> (excuse me, I haven't used hg in a few years)
<cfbolz> pfff
<cfbolz> yes, that's ok
<cfbolz> exarkun: you can just commit that to py3.5, if it fixes things
<exarkun> cool
<exarkun> (I don't remember if I actually have permission to do that or not, but it's entirely possible)
<cfbolz> exarkun: what's your bitbucket account? exarkun?
<exarkun> yes
<fijal> exarkun: hi JP
<cfbolz> exarkun: yep, you have push rights
<fijal> want to pick something from that list? ;-)
<exarkun> yea, saw a couple things that might be easy starting places
<exarkun> just getting my development environment in shape now, going to try to reproduce something locally
<exarkun> also, trying to trick lastpass into remembering my bitbucket password
<fijal> you don't need to translate FYI
<fijal> (at least not for tests outside of lib-python)
<cfbolz> exarkun: cool, shout if you need help with anything
<fijal> we should probably also run this on 64bit linux
Rhy0lite has quit [Quit: Leaving]
Ryanar has quit [Quit: Ryanar]
<ronan> exarkun: BTW, in a case like this, if it passes annotation then it's good to go
<exarkun> okay cool, guess it's good then
<ronan> ah, maybe not, actually, since it touches the jit
rokujyouhitoma has joined #pypy
<kenaan> exarkun py3.5 22c27ed7a494 /pypy/objspace/std/bytesobject.py: encode the type name to bytes for interpolation fixes annotator error: string formatting mixing strings and uni...
<exarkun> out of time for today :) ttyl
rokujyouhitoma has quit [Ping timeout: 260 seconds]
__Myst__ has left #pypy ["WeeChat 1.8"]
dash is now known as ndash
Ryanar has joined #pypy
<arigato> exarkun: oups, thanks for the fix
mattip has joined #pypy
<arigato> ah, I made the change to default (where it is correct) and then just merged into py3.5
<ronan> arigato: BTW, I didn't really understand the explanation, what do you mean by "single jit loop"?
rokujyouhitoma has joined #pypy
<kenaan> mattip default 97134fb162c0 /pypy/doc/whatsnew-head.rst: document merged branches
lritter has quit [Quit: Leaving]
rokujyouhitoma has quit [Ping timeout: 260 seconds]
mattip has left #pypy ["bye"]
<arigato> ronan: the appexec() creates *one* function that is called from all places in the program that require it
<arigato> so if the program happens to use bytearray.extend(x) on various types of 'x' from various unrelated places, these types will all get mixed up in the single function
<ronan> so what? we'll just get guards
<ronan> * and bridges
<arigato> yes, but there are downsides
<arigato> this just makes one loop with a number of bridges
<arigato> so for example, it will keep alive all types ever seen
<arigato> (also, sometimes, bridges are less efficient than straight loops, but in this case it's likely not much of a problem because of loop unrolling)
<ronan> wait, jitted loops keep all the types alive?
<arigato> yes, a guard_class(p5, X) keeps the type object X alive
<ronan> hmm, that's a bit annoying
<arigato> maybe it shouldn't, and also maybe we should try to finish guard_compatible(), but these are not done
hawkowl has quit [Max SendQ exceeded]
<arigato> indeed, maybe guard_value(p5, ConstPtr) shouldn't keep the ConstPtr alive, but detect when that ConstPtr really dies and patch the guard_value into a guard-always-fails
hawkowl has joined #pypy
<arigato> (sorry, that was about guard_value, not guard_class, because guard_class is about an RPython class)
<arigato> but doing it correctly is more involved
<arigato> for example, the trace might contain guard_value(p5, ConstPtr1) and then use ConstPtr2, which was read out of ConstPtr1
<arigato> when we turn the guard_value into a guard_always_fails, we'd need to make the rest of the trace dead too, from the point of view of not keeping its constptrs alive
<arigato> but it needs to be done super-carefully
<arigato> so far we never attempt to free parts of a whole loop+bridges; we only free the whole loop+bridges as a unit
<arigato> anyway, guard_compatible looks like it would provide the correct solution, again
Ryanar has quit [Quit: Ryanar]
Ryanar has joined #pypy
Ryanar has quit [Client Quit]
<kenaan> rlamy cpyext-leakchecking ee6ee454da80 /pypy/module/cpyext/test/: Create builtin pyobjs early for interp-level tests as well
<kenaan> rlamy cpyext-leakchecking 5e01a8ace2de /pypy/module/cpyext/test/test_traceback.py: fix pseudo-leak in test_traceback.py
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 260 seconds]
vkirilichev has joined #pypy
adamholm_ has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholm_ has joined #pypy
adamholm_ has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 240 seconds]
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 240 seconds]
oberstet has quit [Ping timeout: 260 seconds]
forgottenone has quit [Ping timeout: 260 seconds]
vkirilichev has quit [Remote host closed the connection]
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 260 seconds]
<kenaan> wlav default edf12627beaf /pypy/module/cppyy/test/test_cint.py: remove leftover test_cint.py (and higher directories)
yuyichao_ has joined #pypy
yuyichao has quit [Ping timeout: 260 seconds]
<Cheery> Today I had to lug around 64-bit C values.
<Cheery> Although those were pointer-like, so it would have been possible to lug around opaque values.
<Cheery> anyway, it's starting to motivate the setup of the bignum stack into Lever
https_GK1wmSU has joined #pypy
https_GK1wmSU has left #pypy [#pypy]
<kenaan> tobweber stmgc[c8-tcp-style-trx-length] dba308a7d960 /c8/stm/nursery.c: Set exponential base to two, i.e., double trx length on commit
<kenaan> tobweber stmgc[c8-tcp-style-trx-length] d86628e2626c /c8/stm/nursery.c: Increase min trx length and fix capping trx length
<kenaan> tobweber stmgc[c8-tcp-style-trx-length] c6265dd2c77c /c8/stm/: Distinguish min and default trx length to allow shrinking to single instruction level
<kenaan> tobweber stmgc[c8-tcp-style-trx-length] 08a8d7fd1866 /c8/stm/: Backed out changeset c6265dd2c77c
<kenaan> tobweber stmgc[c8-tcp-style-trx-length] 70e630a22710 /c8/stm/nursery.c: Decrease trx len just ten percent at a time below min
<kenaan> tobweber stmgc[c8-tcp-style-trx-length] d9897d451fff /c8/stm/nursery.c: Disable slow start
<kenaan> tobweber stmgc[c8-tcp-style-trx-length] 5563999ed658 /c8/stm/core.c: Backed out changeset dbea548c4c6e
<kenaan> tobweber stmgc[c8-tcp-style-trx-length] feed32340eb2 /c8/stm/nursery.c: Grow backoff cumulatively on repeated aborts
<kenaan> tobweber stmgc[c8-tcp-style-trx-length] 48e819b53680 /c8/stm/nursery.c: Backed out changeset feed32340eb2
<kenaan> tobweber stmgc[c8-tcp-style-trx-length] 6a3c6e601284 /c8/stm/nursery.c: Set backoff to best working value so far and re-enable slow start
<kenaan> tobweber stmgc[c8-tcp-style-trx-length] 3bc9bfa7d481 /c8/stm/: Initialize trx len roughly to old default of 1MB and hard cap on lower limit of 4KB
<kenaan> tobweber stmgc[c8-tcp-style-trx-length] 2828bbba12a4 /c8/stmgc.h: Merge warm up complete event
marr has quit [Ping timeout: 260 seconds]