cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | the secret reason for us trying to get PyPy users: to test the JIT well enough that we're somewhat confident about it
drolando has joined #pypy
dddddd has quit [Remote host closed the connection]
tayfun26 has quit [Remote host closed the connection]
<arigato>
jdeuce: can you expand on how you would use PyModule_GetState() to access C-level globals? I don't quite follow
<arigato>
if you're talking about refactoring the C code anyway, then with CFFI you should refactor it in a way that makes sense for C: how would you make it thread-safe in C? Maybe by moving all globals inside some struct, and changing all functions to take a pointer to such a struct. Then do the same with CFFI
<arigato>
and then, I'd recommend to wrap that struct in a Python class, so that multiple threads in your program each make one class and use that
<arigato>
or if you don't want that, write Python code that fetches the struct pointer from Python-level thread locals (_thread.threadlocals())
<arigato>
(you can do that in C, too, but it's a bit more involved to get portable thread-local structs)
<trfl>
mattip_: is there any documetation on that? I've been traversing the documentation but couldn't find any concrete examples/explanations
<trfl>
and I'm gonna run through all tests at least once just in case, considering it's a new distro :>
hotpot33 has quit [Ping timeout: 260 seconds]
<trfl>
actually, are you saying tgere is no way to test a finished build of pypy2/3? and the only way is to run rpython under python2 and do the translation at runtime while testing?
hotpot33 has joined #pypy
jacob22__ has joined #pypy
<cfbolz>
trfl: you can run the CPython tests on top of the compiled pypy
<cfbolz>
trfl: pypy3 -m test
dustinm has quit [Quit: Leaving]
dustinm has joined #pypy
hotpot33 has quit [Ping timeout: 260 seconds]
marky1991 has joined #pypy
CountryNerd has joined #pypy
CountryNerd_ has quit [Ping timeout: 240 seconds]
hotpot33 has joined #pypy
marky1991 has quit [Ping timeout: 240 seconds]
<trfl>
sweet, thanks! does that run all tests including the pypy specific ones?
stevenja_ has joined #pypy
adamholmberg has joined #pypy
Rhy0lite has joined #pypy
marky1991 has joined #pypy
realitix has quit [Ping timeout: 248 seconds]
CountryNerd has quit [Quit: CountryNerd]
<cfbolz>
trfl: no, that just runs the ones from the CPython stdlib
<cfbolz>
(aside: if you add my nickname to your messages I'll reply more quickly ;-))
stevenja_ has quit [Remote host closed the connection]
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
stevenja_ has joined #pypy
marky1991 has quit [Quit: Saliendo]
<trfl>
aight well i suppose that should cover most common issues you might run into... but i'm curious whether the jit works correctly on our hipster environment :>
<trfl>
(and i'm stuck at work anyways so some latency is fine :p)
<the_rat>
Do nightly builds include debuggy stuff that is not there in release builds?
andrewsmedina has joined #pypy
andrewsmedina has quit [Client Quit]
<cfbolz>
the_rat: I don't remember, maybe they aren't stripped?
andrewsmedina has joined #pypy
<the_rat>
That would be fine
<cfbolz>
trfl: pypy3 -m test.pystone
<the_rat>
I'm testing this app with PyPy and memory usage is growing and groooowing n times what it was with CPython
<the_rat>
So wanted to exclude the obvious causes, like debug tracing inside the interpreter
<cfbolz>
the_rat: no no, I'll merge the fix and kick of a build for you ;-)
<the_rat>
Oh yes, that could definitely affect my code
<the_rat>
That would be amazing
Taggnostr3 has joined #pypy
Taggnostr2 has quit [Ping timeout: 248 seconds]
<jdeuce>
arigato: i'm hoping to maintain compatibility with other C programs that use the old code single threaded. with PyModule_getstate I could just have one global struct that contains all the other globals, and then change all the old globals to be macros that reference the new value in the global struct
<kenaan>
mattip default c362e84211a7 /pypy/doc/sandbox.rst: add warning for sandbox
<cfbolz>
the_rat: sorry, need to run now, but will try to get that done later
<the_rat>
cfbolz: that was very helpful, thanks a lot again! No hurry :)
<jdeuce>
arigato: so when each thread starts up python would allocate another global struct copy of it, and then the macro to access old globals would be something like #define foo (struct GlobalStruct*)PyModule_GetState(module)->foo;
igitoor has quit [Read error: Connection reset by peer]
mattip has quit [Ping timeout: 260 seconds]
igitoor has joined #pypy
<arigato>
jdeuce: as a general rule, what you want for cffi is change your C code so that it remains usable as C code, too, but this includes not calling python-specific functions
<arigato>
so it means you need to do the same trick not using python-specific functions
<arigato>
thread-locals are a platform mess, but if you only care about (say) modern Posix with gcc, then you can use "__thread"
marky1991 has joined #pypy
<arigato>
if you care about older Posix, then pthread_getlocal() I think (I may misremember the exact function name)
<arigato>
on Windows it's another function
<arigato>
you may find some headers that do the correct dance of #ifdefs
<arigato>
e.g. you can copy the files rpython/translator/c/src/thread*.h
<arigato>
and adapt
<arigato>
at the end of the day, cffi doesn't give you help with that, because that is supposed to be a problem that needs solving in pure C
igitoor has quit [Changing host]
igitoor has joined #pypy
<jdeuce>
arigato: yeah that's why I want to just use the PyModule_getstate stuff, then I can bury the refactor in the C code behind a #PYTHON_THREADLOCAL ifdef or something
jcea has quit [Quit: jcea]
<arigato>
that's not going to happen: no PyXxx function can be sanely called from cffi C (for example, you have no GIL, you don't control exceptions, and in PyPy it would need different hacks)
<arigato>
in other words, my point of view is that calling any function from the CPython C API from cffi C is equivalent to: you have some program written in pure C, and at some point, for convenience, you want to call a function from the CPython C API
<arigato>
you can't do that (in this example, you'd need much more messy steps to initialize the Python interpreter, etc.; it's not meant to be used that way)
rubdos has quit [*.net *.split]
wallet42 has quit [*.net *.split]
DRMacIver has quit [*.net *.split]
mwhudson has quit [*.net *.split]
yuvipanda has quit [*.net *.split]
thnee has quit [*.net *.split]
mdash has quit [*.net *.split]
jiffe has quit [*.net *.split]
jacob22__ has quit [Quit: Konversation terminated!]
rubdos_ has joined #pypy
jacob22__ has joined #pypy
<arigato>
if you don't want to write messy-but-pure-C logic, you could alternatively declare @ffi.def_extern() callbacks
<arigato>
"#define foo _get_tl_foo()"
<jdeuce>
hmm
<arigato>
and _get_tl_foo() is declared as a function implemented in Python as a def_extern
<arigato>
you'd basically be implementing _get_threadlocal() in normal Python, maybe using the _thread.threadlocal class
yuvipanda has joined #pypy
jiffe has joined #pypy
mdash has joined #pypy
jiffe has quit [Max SendQ exceeded]
<arigato>
note that if you're not really worried about threads, but only CPython subinterpreters, then you can likely just declare _get_threadlocal() as "return p_global"
<arigato>
because there is logic anyway that will call the correct _get_threadlocal(), i.e. the one defined in the current subinterpreter
jiffe has joined #pypy
dash has quit [Ping timeout: 245 seconds]
bendlas has quit [Ping timeout: 240 seconds]
yuvipanda has quit [Ping timeout: 240 seconds]
danieljabailey has quit [Ping timeout: 240 seconds]
agates has quit [Ping timeout: 255 seconds]
abvi[m] has quit [Ping timeout: 256 seconds]
oberstet has joined #pypy
danieljabailey has joined #pypy
jacob22__ has quit [Ping timeout: 268 seconds]
stevenja_ has quit [Remote host closed the connection]
stevenjackson121 has joined #pypy
stevenjackson121 has quit [Ping timeout: 245 seconds]
stevenjackson121 has joined #pypy
ceridwen has quit [Ping timeout: 265 seconds]
Taggnostr2 has joined #pypy
Taggnostr3 has quit [Ping timeout: 240 seconds]
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
Taggnostr3 has joined #pypy
Taggnostr2 has quit [Ping timeout: 256 seconds]
stevenjackson121 has quit [Remote host closed the connection]
stevenjackson121 has joined #pypy
stevenjackson121 has quit [Read error: Connection reset by peer]
stevenjackson121 has joined #pypy
<cfbolz>
mattip_: would you have time to merge default to py3.5? I tried quickly but ran into trouble with INTP_real, because that problem has already been solved in a slightly different way on py3.5
bendlas has joined #pypy
marky1991 has quit [Read error: Connection reset by peer]
marky1991 has joined #pypy
Hasimir has joined #pypy
stevenjackson121 has quit [Remote host closed the connection]