cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end ) | use cffi for calling C | if a pep adds a mere 25-30 [C-API] functions or so, it's a drop in the ocean (cough) - Armin
ronan has quit [Ping timeout: 272 seconds]
<bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/6834 [antocuni: force build, rgil-track-thread]
<bbot2> Failure: http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/6834 [antocuni: force build, rgil-track-thread]
<bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-32/builds/5749 [antocuni: force build, rgil-track-thread]
<bbot2> Failure: http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-32/builds/5749 [antocuni: force build, rgil-track-thread]
gracinet has quit [Quit: Leaving.]
<bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-jit-macosx-x86-64/builds/4935 [antocuni: force build, rgil-track-thread]
<bbot2> Failure: http://buildbot.pypy.org/builders/pypy-c-jit-macosx-x86-64/builds/4935 [antocuni: force build, rgil-track-thread]
<bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/5154 [antocuni: force build, rgil-track-thread]
<bbot2> Failure: http://buildbot.pypy.org/builders/rpython-win-x86-32/builds/249 [antocuni: force build, rgil-track-thread]
<bbot2> Failure: http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/5154 [antocuni: force build, rgil-track-thread]
oberstet has joined #pypy
adamholmberg has joined #pypy
dddddd has quit [Read error: Connection reset by peer]
adamholmberg has quit [Ping timeout: 258 seconds]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 258 seconds]
tsaka__ has joined #pypy
xcm has quit [Read error: Connection reset by peer]
xcm has joined #pypy
Ai9zO5AP has quit [Quit: WeeChat 2.7.1]
oberstet has quit [Ping timeout: 256 seconds]
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
oberstet has joined #pypy
marvin has quit [Remote host closed the connection]
marvin_ has joined #pypy
marvin_ has quit [Remote host closed the connection]
marvin has joined #pypy
jvesely has quit [Quit: jvesely]
marvin has quit [Remote host closed the connection]
marvin has joined #pypy
lazka has quit [Quit: bye]
lazka has joined #pypy
gracinet has joined #pypy
tsaka__ has quit [Ping timeout: 272 seconds]
suhdonghwi has joined #pypy
gracinet has quit [Quit: Leaving.]
<suhdonghwi> I tried to find what `_attrs_` (RPython) is but it seems that there is no documentation for this property. What is this and how does it help optimization?
<cfbolz> suhdonghwi: it's sort of like __slots__ in general python
<cfbolz> it's a declaration of what fields are allowed in a class
gracinet has joined #pypy
Ashleee has quit [Remote host closed the connection]
Ashleee has joined #pypy
suhdonghwi has quit [Remote host closed the connection]
Guest33 has joined #pypy
ronan has joined #pypy
fangerer_ has joined #pypy
suhdonghwi has joined #pypy
tsaka__ has joined #pypy
ronan has quit [Remote host closed the connection]
ronan has joined #pypy
ronan has quit [Remote host closed the connection]
ronan has joined #pypy
suhdonghwi has quit [Remote host closed the connection]
lritter has joined #pypy
suhdonghwi has joined #pypy
ronan has quit [Remote host closed the connection]
ronan has joined #pypy
<suhdonghwi> I tried using `virtualizable` feature in my JIT code. It builds successfully without error but it shows this warning: `ignoring hint {'access_directly': True, 'fresh_virtualizable': True}` Is this warning safe to ignore?
<cfbolz> suhdonghwi: yes, that warning can sometimes happen without it being a problem
<cfbolz> suhdonghwi: is your code online somewhere?
<suhdonghwi> it is `Frame` class which i tested virtualizable feature
<cfbolz> suhdonghwi: right, that's the intended use
<cfbolz> suhdonghwi: does it help performance?
<suhdonghwi> No, it does not. It rather decreases performance.
<cfbolz> then you need to start looking at traces
<cfbolz> do you know how to get traces from the JIT?
<suhdonghwi> yes, but what part should i look at to check if virtualizable feature is working?
<cfbolz> suhdonghwi: if you compare traces before and after, there should be no operations reading from the frame any more
<cfbolz> suhdonghwi: ah, the stack should maybe be also on the frame, and also be immutable. but that means you need to make it fixed-size and compute the maximum stack depth
<suhdonghwi> aha, that maybe would help performance. i'll try it. thank you!
xcm has quit [Ping timeout: 255 seconds]
xcm has joined #pypy
<cfbolz> suhdonghwi: I just built your vm, any program I should try?
<cfbolz> ah, there is a checked in log file
<cfbolz> I looked at that, it seems to do stack handling
<cfbolz> all these kinds of calls:
<cfbolz> cond_call(i25, ConstClass(_ll_list_resize_hint_really_look_inside_iff__listPtr_Signed_Bool), p5, i23, 1, descr=<Callv 0 rii EF=5>)
<suhdonghwi> what is a stack handling?
<cfbolz> suhdonghwi: all the stack.append(...) and stack.pop() calls in the Interpreter.run method
<cfbolz> so if you do what I said above things should become faster: (move the stack into the frame, use a fixed-size list and precompute the maximum stack depth)
<cfbolz> makes sense?
<suhdonghwi> hmm, i am trying to understand, what does `cond_call(i25, ConstClass(_ll_list_resize_hint_really_look_inside_iff__listPtr_Signed_Bool), p5, i23, 1, descr=<Callv 0 rii EF=5>` )
<cfbolz> suhdonghwi: that's what the append and the pop turns into
<cfbolz> this is an append, for example: https://www.irccloud.com/pastebin/IdoK8Wrz/
tsaka__ has quit [Read error: Connection reset by peer]
tsaka__ has joined #pypy
<suhdonghwi> ah, i got it. and.. how does virtualizable feature help that operation run faster?
<cfbolz> suhdonghwi: the first step is to stop using list.append and list.pop for your stack
<cfbolz> the idea is that you maintain an index into a fixed-sized list that tells you how deep the stack is currently
<suhdonghwi> so that i can avoid frequent resizing of the stack list?
<cfbolz> suhdonghwi: every function has a maximum stack usage, right?
<suhdonghwi> yes they do
<cfbolz> suhdonghwi: right, so you initialize the stack to [None] * max_stack_depth
<cfbolz> and then you keep the current stack_depth as an integer
ronan has quit [Remote host closed the connection]
ronan has joined #pypy
<cfbolz> suhdonghwi: give me a couple of minutes, I can sketch you some code
<suhdonghwi> thank you very much for kind help ..!
<cfbolz> cheers :-)
<cfbolz> suhdonghwi: is there an example program I can run?
<suhdonghwi> this code calculates 30th fibonacci number and prints it
gracinet has quit [Quit: Leaving.]
<cfbolz> suhdonghwi: cool
ronan has quit [Read error: Connection reset by peer]
ronan has joined #pypy
<cfbolz> (I'm a huge fan of non English programming languages)
<suhdonghwi> oh you noticed it :p using Rpython for this project is really fun experience i think
<cfbolz> suhdonghwi: hm, the test.h doesn't actually print anything for me
tsaka__ has quit [Ping timeout: 255 seconds]
<cfbolz> but the code you pushed in 46c62ac4cd5fa5c380d3675e989ee8d11e4adfca looks good
suhdonghwi_ has joined #pypy
<suhdonghwi> oh, i just JIT-built it now and it got significant performance boost!
<cfbolz> can you update the log file for me?
<suhdonghwi> sure, here it is: https://pastebin.com/iMDypZMQ
jcea has joined #pypy
<cfbolz> suhdonghwi: Frame.load should not be elidable
<suhdonghwi> ohh, that's right i forgot to comment it out
<suhdonghwi> but what actually happens when i mark not-elidable function as elidable?
<cfbolz> suhdonghwi: the code will behave incorrectly
<cfbolz> suhdonghwi: I opened a pull request that fixes a problem in the virtualizable declaration
gracinet has joined #pypy
fangerer_ has quit [Quit: Connection closed for inactivity]
<suhdonghwi> this helps me a lot! i really appreciate it
<suhdonghwi> i'll check and study it now thank you again!
<cfbolz> cheers
<cfbolz> there's still stuff to be improved ;-)
<cfbolz> suhdonghwi: but if you look at the trace now, some bytecodes produce zero operations
<suhdonghwi> hmm, does that mean it is optimized away?
<cfbolz> yes
<suhdonghwi> well it definitely is a huge leap of an optimization :) i'll keep try to optimize it more
ronan has quit [Remote host closed the connection]
ronan has joined #pypy
<cfbolz> suhdonghwi: further things to look at: the base class Constant should have no attributes! Right now ConstantInteger also has an unused 'doubleval' field (etc)
<cfbolz> suhdonghwi: also, calls are really expensive right now, with lots of list manipulation going on
xcm has quit [Remote host closed the connection]
<suhdonghwi> ah yes, calls are very expensive. that is because the language i am currently uses named parameter and partial application by default
<suhdonghwi> i think i should optimize that part harder
<cfbolz> that's not a problem as such
<cfbolz> but the current implementation needs improvement
xcm has joined #pypy
<suhdonghwi> yes! and about Constant class' attributes,
<suhdonghwi> i added `_attrs_` to Constant because when i remove it it shows an error like
<suhdonghwi> `ConstInteger has _attrs_, but not its base class`
<suhdonghwi> how should i deal with this error correctly?
<cfbolz> suhdonghwi: the _attrs_ of Constant should be ['type'], I suppose?
tsaka__ has joined #pypy
ronan has quit [Remote host closed the connection]
tsaka__ has quit [Ping timeout: 255 seconds]
ronan has joined #pypy
<suhdonghwi> hmm, there's a code where it accesses fields such as `intval` or `doubleval` without downcasting(?) Constant object. so compilation error is thrown at that point.
<cfbolz> suhdonghwi: you downcast by saying "assert isinstance(obj, ConstantInteger)"
<cfbolz> suhdonghwi: or you could replace the "other.type == TYPE_INTEGER" tests by "isinstance(other, ConstantInteger)"
<suhdonghwi> oh, is that conditional statement automatically recognized by RPython translator?
<cfbolz> yes
<suhdonghwi> wow, compiler is smarter than i thought
xcm has quit [Remote host closed the connection]
<cfbolz> suhdonghwi: you might not need a .type attribute at all
xcm has joined #pypy
<suhdonghwi> but isn't `isinstance` checking slower than comparing type with constant?
adamholmberg has joined #pypy
<cfbolz> suhdonghwi: no
<cfbolz> it compiles to the same thing
<suhdonghwi> aha, i got it
<suhdonghwi> i
ronan has quit [Read error: Connection reset by peer]
ronan has joined #pypy
<suhdonghwi> i really could remove type attribute completely!
<cfbolz> good
ronan has quit [Remote host closed the connection]
ronan has joined #pypy
fangerer_ has joined #pypy
ronan has quit [Remote host closed the connection]
ronan has joined #pypy
suhdonghwi has quit [Remote host closed the connection]
adamholmberg has quit [Remote host closed the connection]
jvesely has joined #pypy
adamholmberg has joined #pypy
Rhy0lite has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
jcea has quit [Quit: jcea]
jacob22 has quit [Read error: Connection reset by peer]
jacob22 has joined #pypy
inhahe has joined #pypy
inhahe_ has quit [Read error: Connection reset by peer]
jcea has joined #pypy
jcea has quit [Quit: jcea]
<Dejan> pandas gives me a warning that it could not import lzma - (pypy-c-jit-98800-010181c29a26-linux64)
<Dejan> nothing critical, I was just wondering whether it is a known issue ?
<mattip> That is a pypy2.7 build right? lzma was only added in python3.3
<Dejan> nope, this is 3.6.9
<mattip> hmm. Can you import lzma? It seems to be broken on the portable builds
<Dejan> i see _lzma* stuff...
<Dejan> in the lib_pypy
<mattip> I get undefined symbol: lzma_index_stream_padding
<cfbolz> arigato: I added __pypy__.utf8content for unicode debugging
<Dejan> mattip, same here
<Dejan> I will check with the 3.7 build i have installed
<Dejan> no luck, same error
<Dejan> anyway, just wanted to let you know
<Dejan> I do not use lzma anywhere in my code
<Dejan> so I am fine
<Dejan> also, the nightly build I am using is few weeks old
<Dejan> I will check the latest one right now
<mattip> nothing will change. It is a build configuration problem
<Dejan> ah, OK
<Dejan> perhaps a test that just imports standard modules should be in the test suite
dddddd has joined #pypy
jcea has joined #pypy
bitbit has joined #pypy
Smigwell has joined #pypy
Dejan has quit [Ping timeout: 255 seconds]
Dejan has joined #pypy
Ai9zO5AP has joined #pypy
Guest33 has quit [Remote host closed the connection]
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
jacob22 has quit [Read error: Connection reset by peer]
glyph has quit [Excess Flood]
glyph has joined #pypy
jacob22 has joined #pypy
oberstet has quit [Remote host closed the connection]
jcea has quit [Remote host closed the connection]
jcea has joined #pypy
blueyed has quit [Quit: WeeChat 1.5]
Rhy0lite has quit [Quit: Leaving]
Ai9zO5AP has quit [Ping timeout: 256 seconds]
lritter has quit [Quit: Leaving]
Ai9zO5AP has joined #pypy
Smigwell has quit [Ping timeout: 255 seconds]
kipras has quit [Read error: Connection reset by peer]
tsaka__ has joined #pypy
tsaka__ has quit [Ping timeout: 272 seconds]
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
fangerer_ has quit [Quit: Connection closed for inactivity]
adamholmberg has quit [Remote host closed the connection]
jacob22 has quit [Read error: Connection reset by peer]
jacob22 has joined #pypy