09:16 <smvv> is it possible to see what rpython IR opcodes correspond to what asm x64 code? I'm trying to understand why certain asm code is generated. I found MARK_TRACE_ASM but I'm not sure how I can enable that and if that will print a trace of the rpython IR with the generated asm next to it

09:17 <smvv> I've set PYPYLOG=jit:pypy.log and disassembled the code in jit-backend-dump using gobjdump

09:18 <smvv> I found that some of the generated instructions comes from flush_cc() but the preceding code is a still mystery since many guards can call flush_cc()

09:28 <fijal> smvv: you can find the corresponding addresses in jit log

09:28 <smvv> jit-backend-addr?

09:30 <fijal> yeah

09:31 <fijal> still trying to paste a bunch of stuff :-/

09:32 <fijal> but yes, jit-backend-addr maps the loop to the address in assembler

09:32 <fijal> we put some effort into making vmprof put it all together, but has proven to be hard

10:01 <smvv> fijal: I see the mapping you're mentioning but I was looking for a more fine grained mapping (addresses to rpython IR nodes)

10:12 <fijal> that's I think gettable from indexes

10:12 <fijal> so on jit-log-backend you have offsets from beginning to the IR

10:13 <fijal> https://paste.pound-python.org/show/38RR1FlZC5wAI5TfPPyy/

10:13 <fijal> on line 397 you have +277 - this is 277 bytes offset from the beginning

10:23 <smvv> fijal: awesome!

10:24 <fijal> we have some infrastructure to do it for you

10:24 <fijal> but it's a bit unclear how to use it all

10:26 <smvv> fijal: you mean jitviewer or vmprof?

10:33 <cfbolz> smvv: yes, though it's all in various stages of bitrot :-(

14:12 <RemoteFox> hola

14:13 <simpson> Hi.

14:19 <RemoteFox> what data type would say is best for bytecodes in my language implemented in rpython?

14:19 <RemoteFox> so far I am using list with ints

14:20 <RemoteFox> but as I have only a few instructions, I thought it would be better to use something immutable with uints / bytes

14:20 <fijal> yes, immutable is much better

14:20 <fijal> we usually use strings and read byte by byte

14:20 <fijal> makes it more compact, but also you don't have to give extra hints to constant fold away opcode reading when doing JIT

14:20 <fijal> RemoteFox: ^^^

14:21 <RemoteFox> hah

14:21 <RemoteFox> strings didn't occured to me

14:21 <RemoteFox> I was thinking more about bytearray or something from https://rpython.readthedocs.io/en/latest/rtyper.html?highlight=struct#array-types

14:21 <RemoteFox> but strings make perfect sense

14:21 <RemoteFox> thanks

19:51 <lesshaste> is it hard to use pypy with jupyter notebooks?

19:56 <lesshaste> is it just this?

19:56 <lesshaste> pypy -m pip install ipykernel

19:56 <lesshaste> pypy -m ipykernel install --name pypy --display-name="Python (PyPy)"

20:04 <smvv> Is it possible that the jump targets of guards are not filled in when the asm x64 code is dumped in the jit-backend-dump section? I'm seeing guards encoded as jumps to the next instrunction instead of going elsewhere on failure https://www.irccloud.com/pastebin/Qd8etWdp/

20:06 <smvv> the guards corresponding to those jumps are: https://www.irccloud.com/pastebin/DIJz8r1K/

22:08 <energizer> pypy produces some information about the execution, and then uses that information to make it faster on successive executions of the same part of the code. right?

22:21 <simpson> energizer: That's a way of looking at it. What PyPy does is slightly more specific: After a loop has been run many times, PyPy records what happens during one particular run of the loop, and uses that information to do some specialization. That cycle of recording and specializing might happen several times. The hope is that, after specializing, there will be fewer operations.

22:27 <energizer> simpson: ive heard informally that pypy is slower at the beginning of execution than cpython is. i take it that slowness is because it's taking time to record things?

22:29 <energizer> (and to produce the specialized version of the loop)

22:31 <tos9> AFAIK only partially

22:31 <tos9> energizer: even if you turn the JIT off, AFAIR, pypy is a bit slower than cpython

22:33 <Ulfalizer> energizer: yeah, i see that effect pretty clearly in my program (hand-rolled parsing thing). anecdotally, pypy needs a runtime of at least a few seconds before it overtakes cpython in it.

22:34 <energizer> would it be possible to persist that recorded info and use it in another run of the program?

22:35 <tos9> energizer: (I don't know what the reasoning is, but presumably it's either that the C that is generated by the rpython backend is slower than the C written by hand for CPython, or that the actual interpreter architecture is more complex. I'd guess possibly the latter, but this is all academic since yeah you use pypy to use the jit :)

22:35 <tos9> energizer: if you mean for jitted pypy, AFAIK doing it is super hard (because of garbage collection)

22:35 <tos9> but yes in theory it's possible.

22:36 <energizer> i see

22:39 <energizer> does pypy keep getting faster with longer runtimes, by jitting more and more of the program?

22:40 <tos9> energizer: it can, but in modern pypys the jit turns off after awhiel.

22:40 <tos9> (intentionally)

22:41 <simpson> energizer: IME it doesn't make sense to do blanket comparisons of speed between PyPy and CPython, but rather to profile and tune for a specific application.

22:42 <energizer> theoretically you could run your program in multiple process, and then share what they learned with each other for later program runs or even the current program runs?

22:43 <tos9> sort of, you run into the same problem as before

22:43 <tos9> (that the gc, and the jit in general, emits code with hard addresses in it)

22:43 <tos9> so right now no you can't

22:43 <tos9> in theory maybe possibly

22:43 <simpson> energizer: In theory, sure; in practice, PyPy's JIT doesn't do any kind of sharing or remembering. It's a hard problem in general.

22:43 <energizer> mhmm ok