cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | the secret reason for us trying to get PyPy users: to test the JIT well enough that we're somewhat confident about it
<techalchemy>
hey the vmprof-server site says to pop over here for questions -- i can't currently post things to there and was wondering if this is a known issue
Hasimir has quit [Read error: Connection reset by peer]
TheAdversary has quit [Read error: Connection reset by peer]
<maxxam>
PyPy keeps segfaulting. I know there’s a point in my code (somewhere) where even CPython occasionally gets stuck, but (thanks to a watchdog) CPython recovers. I guess PyPy doesn’t. Is there any way to do something like this for debugging it? https://gist.github.com/toolness/d56c1aab317377d5d17a
<maxxam>
* as far as I know I’m not using any 3rd party C modules
wleslie has joined #pypy
<cfbolz>
maxxam: and I assume you have no way to reproduce the segfault in a non production setting?
wleslie has quit [Ping timeout: 256 seconds]
jcea has joined #pypy
phlebas has quit [Quit: Connection closed for inactivity]
altendky has joined #pypy
adamholmberg has joined #pypy
wleslie has joined #pypy
adamholmberg has quit [Ping timeout: 265 seconds]
marself has quit [Ping timeout: 256 seconds]
<idnar>
Note that your python interpreter (whether CPython or pypy-c) may not have been linked by the C++ compiler. This can lead to problems during loading of C++ libraries and program shutdown. In that case, re-linking is highly recommended.
<idnar>
what exactly is "by the C++ compiler" here?
<maxxam>
cfbolz: That’s incorrect, if only on a technicality. The segfault reproduces quite reliably but ONLY in a production setting. Steps to reproduce: 1) Start the program 2) Wait 24-72 hours.
Rhy0lite has joined #pypy
jamesaxl has joined #pypy
DIRT has quit [Ping timeout: 256 seconds]
<Cheery>
cfbolz: I recently looked into Icon language and realised its implementation is a bit prolog-like.
<cfbolz>
maxxam: one thing to try is to see whether it's a JIT bug or not. If you run pypy with --jit off you could see whether it crashes. That's quite a bit slower though :-(
<cfbolz>
Cheery: sort of, yes
<Cheery>
cfbolz: I did read few papers from pyrolog because of this, but I wonder, how would you implement pyrolog's interpreter today if you did?
<maxxam>
cfbolz: right, err, is it slower than CPython?
<cfbolz>
maxxam: yes
adamholmberg has joined #pypy
<maxxam>
cfbolz: hmm, the whole reason I’m trying PyPy is CPython was too slow for the server
<cfbolz>
maxxam: right, just saying that that's a good piece of knowledge to find out for debugging the segfault
<cfbolz>
maxxam: the other thing you should do is build pypy with lldebug turned on. That gives you debugging symbols and enables a number of extra asserts. Where did you get your executable from?
<maxxam>
cfbolz: ok, noted.
<maxxam>
cfbolz: the pypy executable came from the website
<maxxam>
you’re probably right I will need to build it with lldebug
<maxxam>
…so I am looking at https://pypy.org/download.html#building-from-source and the server this is running on has a total 4GB of RAM. it’s a 64 bit machine. The documentation suggests that building PyPy is not going to work. Is that true?
<cfbolz>
maxxam: yes, that probably won't work :-(
<maxxam>
cfbolz: hmm. well I could build it on my macOS desktop (8GB ram) or fire up a VM (5GB vm on 8gb host? ouch!)
<cfbolz>
Sorry :-(
<cfbolz>
Cheery: puh, difficult question. Did you read Laurence Tratt's rpython & converge paper too? That has an icon like expression system
<cfbolz>
Getting prolog really right in rpython is not that trivial, but I am not sure the lessons really transfer
marky1991 has joined #pypy
<maxxam>
cfbolz: ok so assuming I manage to get this compiled on a VM, what info can I actually get out of it
<maxxam>
?
DIRT has joined #pypy
oberstet has joined #pypy
marky1991 has quit [Ping timeout: 256 seconds]
<LarstiQ>
maxxam: a backtrace with symbols
<cfbolz>
maxxam: if you're lucky, then your segfault gets turned into a assertion failure. Otherwise you at least have debug symbols and can try to debug the core dump
<maxxam>
LarstiQ , cfbolz: ok, is there any documentation anywhere describing what happens and what information comes out and how to deal with it?
<cfbolz>
Nope, that's indeed not such a nice experience. But if you paste the stack trace here, will give you a hand
marky1991 has joined #pypy
<maxxam>
cfbolz: ok, thanks. It won’t happen in the next couple of minutes but I know where to ask :)
<cfbolz>
maxxam: yup. Feel free to ping me
<cfbolz>
What does the server do, btw?
<maxxam>
cfbolz: processing data in realtime
<cfbolz>
Scary ;-)
<cfbolz>
maxxam: and you're finding pypy's response times reasonable?
<maxxam>
cfbolz: …and exactly what pypy SHOULD be ideal for :)
<maxxam>
cfbolz: well, switching from cpython to pypy got the load average under 1.0 so that’s goo
<maxxam>
*good
<cfbolz>
Ok
<cfbolz>
maxxam: cool, I suppose ;-)
maxxam_ has joined #pypy
maxxam has quit [Ping timeout: 260 seconds]
maxxam_ is now known as maxxam
oberstet has quit [Read error: Connection reset by peer]
stevenja_ has joined #pypy
oberstet has joined #pypy
stevenja_ has quit [Remote host closed the connection]
stevenja_ has joined #pypy
stevenja_ has quit [Remote host closed the connection]
stevenja_ has joined #pypy
stevenja_ has quit [Remote host closed the connection]
stevenja_ has joined #pypy
stevenja_ has quit [Remote host closed the connection]
stevenja_ has joined #pypy
realitix has quit [Quit: realitix]
marself has joined #pypy
stevenja_ has quit [Remote host closed the connection]
stevenja_ has joined #pypy
stevenja_ has quit [Ping timeout: 265 seconds]
stevenja_ has joined #pypy
<ronan>
I'm wondering, what causes a method call to end up as a call_assembler instead of being inlined in the trace?
<cfbolz>
ronan: the called function is small and has no loop?
<ronan>
cfbolz: it's not small, but similar things do get inlined
<cfbolz>
ronan: there are two things that stop inlining: one is the presence of a loop
<cfbolz>
The other is if a function is called in a trace that gets aborted because it becomes too long, then the called function is marked as not inlinable
<cfbolz>
The second mechanism is a bit brittle
<ronan>
ah, I guess that's what happens here
<cfbolz>
ronan: there is a log category which tells you about which functions are marked as not inlinable, but I forgot what it's called
<cfbolz>
ronan: out of curiosity, what's program is this?
<ronan>
does filter() or similar builtins count as a loop?
<ronan>
some client's code
<cfbolz>
Cool
<cfbolz>
About filter: not entirely sure!
<cfbolz>
ronan: filter is just written on the app-level, it seems
<cfbolz>
so it's like a function with a loop. but that makes the functions that call it themselves inlinable (but the call to filter will turn into a call_assembler)
<Cheery>
cfbolz: I didn't read those, but I'll go check at those.