cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | the secret reason for us trying to get PyPy users: to test the JIT well enough that we're somewhat confident about it
wleslie has quit [Quit: ~~~ Crash in JIT!]
<mattip> TIL: python handles greek sigma title() in a special way since there is a \u03C3 greek small letter sigma
<mattip> as well as \u03C2 GREEK SMALL LETTER FINAL SIGMA
<mattip> but does not do the same for Aribic or Hebrew, that also have ending letters
DIRT has quit [Remote host closed the connection]
Garen_ has joined #pypy
Garen has quit [Ping timeout: 240 seconds]
ctismer has quit [Ping timeout: 240 seconds]
ctismer has joined #pypy
<kenaan> mattip default b42a5efd4312 /pypy/module/: fix translation
dddddd has quit [Remote host closed the connection]
<mattip> also strange - u'\u1ff3' == 8179 "Greek Small Letter Omega With Ypogegrammeni" according to python, chr(8179).upper()
<mattip> is the two-char sequence [937, 921] but unicodedb.toupper(8179) gives me 8188
<mattip> Of course I dont know all this, it is from failing tests
<mattip> and exploring things like "ord(chr(8188).lower().upper())" on python3, which fails BTW
<mattip> ord complains it gets string w/len 2
<mattip> wierd, I don't see where we special case that in pypy3, but the test is passing (test_unicodeobject.test_capitalize)
<mattip> ahh, using unicodedb.toupper_full, not unicodedb.toupper
DIRT has joined #pypy
fake-name has joined #pypy
user24 has joined #pypy
raynold has joined #pypy
<fake-name> is there a tool similar to the gdb plugin for python that provides py-bt?
<fake-name> basically, some way I can attach to a running pypy process, and extract a python stack dump
user24 has quit [Quit: Leaving]
ast_ has joined #pypy
ast_ is now known as Guest57086
<mattip> but maybe the world has moved on enough that it just works, if I remember properly we do not strip debug symbols from
<mattip> the nightly builds
<mattip> fake-name: of course if the JIT is enabled, all bets are off
<fake-name> @mattip - yeah, I saw that
<fake-name> was hoping there was something more
<fake-name> actually, this is a dumb question, if I'm just using the release version from the pypy website, is the jit on be default?
<kenaan> mattip unicode-utf8-py3 f287dec62c4e /pypy/objspace/std/: fixes, start to handle some edge cases
<mattip> yes. you can disable it from the cmmand line "pypy --jit off ..."
<bbot2> Started: http://buildbot.pypy.org/builders/own-linux-x86-64/builds/6851 [mattip: force build, unicode-utf8-py3]
<fake-name> more, unless I turn it off explicitly, it's on
<mattip> correct
<fake-name> was iind of going "wait, is it on if I just run `pypy <script>.py`"
<fake-name> *kind
<fake-name> anyways, derp
<mattip> :)
<mattip> fake-name: if you get anywhere with gdb, please let us know what does and doesn't work
<fake-name> mostly, I was just hoping I could stop having to fall back to cpython to diagnose my multiprocessing teardown issues
<fake-name> also, wheeee, multiprocessing queues block on join unless the queue is empty
* mattip zzz
<mattip> good luck
<fake-name> thx
mattip has quit [Ping timeout: 260 seconds]
inhahe__ has joined #pypy
Guest57086 has quit [Ping timeout: 252 seconds]
inhahe__ has quit [Ping timeout: 240 seconds]
<maxxam> cfbolz: thanks!
Garen_ has quit [Read error: Connection reset by peer]
Garen has joined #pypy
<kristina> i just built pypy i think.
<kristina> but i did an oops.
<kristina> 0x000000000000001d (RUNPATH) Library runpath: [RIGIN/../lib:/usr/local/lib]
<kristina> also i passed --no-shared. and it didn't do --no-shared because it links against things that it should not link against.
<kristina> so i take it, i need to recompile /DEPOT/pypy3-v6.0.0/usession-release-pypy3.5-v6.0.0-0/testing_1 but now i need to actually pass in proper flags instead of random stuff pypy adds which is completely bogus?
<kristina> it appears that the build system is really upset because it clobbers the checkout directory. that's very illegal :(
<kristina> it should not emit any artifact that traverses outside /DEPOT/pypy3-v6.0.0/.
wallet42 has quit [Ping timeout: 240 seconds]
<LarstiQ> kristina: --no-shared at least means to not build _libpypy.so_ as a shared object separate from pypy-c, I'm not sure it means statically linking all dependencies
<kristina> oh so i want --shared and then to decompose it to something that can be statically linked?
<kristina> that did the opposite to what i wanted i guess.
<LarstiQ> kristina: I understand your frustration, but "completely bogus" and "very illegal" may not be the most help-inducing formulations
<kristina> well it's illegal to put things in the non artifact directory unless you're installing in which case it goes into the virtual sysroot.
<LarstiQ> kristina: that's a bit a matter of opinion
<LarstiQ> and it's not, I think, that the pypy team is opposed to working on the build system to make it work like that, but neither is it a priority
<bbot2> Failure: http://buildbot.pypy.org/builders/own-linux-x86-64/builds/6851 [mattip: force build, unicode-utf8-py3]
<LarstiQ> I'm sure patches are welcome, freebsd build is maintained by the community too
<kristina> but it's not, i can ask clang to forcefully redirect output using its VFS schenanigans, but you have your own code generator too, so i have to ask the build system to inject something to deal with things that are unable to handle paths properly (it's either that or get warnings about clobbering the checkout directory or fail instantly).
<LarstiQ> kristina: I'm sorry to be the bearer of bad news, but your environment is so far out the norm that it will be up to you to do the work
<LarstiQ> unless you convince someone else
<kristina> well it's normal to have an artifact directory.
<LarstiQ> kristina: maybe have a look at how say Debian solved that?
<LarstiQ> kristina: there is also https://github.com/squeaky-pl/portable-pypy
<kristina> you seem to have that to an extent. with PYPY_USESSION_DIR. but then somewhere it just starts ignoring it completely.
<LarstiQ> kristina: could be there is a bug too
<LarstiQ> how is it being ignored?
<kristina> checkout directory is clobbered too.
wallet42 has joined #pypy
<kristina> it seems that happens when it compiles modules.
<kristina> rpython itself seems to not clobber.
tayfun26 has joined #pypy
<cfbolz> kristina: yes, I think it should be only cffi modules, pyc files, and then final executable and the .so that get copied to the checkout
<kristina> last run i built it like this: http://crna.cc/mk3wLxpiqqUyJsR
<kristina> the --no-shared is wrong, i misunderstood the flag.
<cfbolz> kristina: some of the options seem redundant, eg --thread --continuation
<cfbolz> Those should be on by default
<cfbolz> Same for all the withmod options
<kristina> what about the jit? also i still don't understand, is pypy a py->c compiler and/or a jit for stuff that's not precompiled or is it one of those?
<kristina> as far as i understand it has an optional JIT to do that that you enable with --opt=jit and that other jit flag.
<fake-name> AIUI, it JITs the bytecode, without a internal C step
<fake-name> and from earlier, the JIT is on unless you specifically turn it off
<kristina> what's --opt=jit?
<fake-name> ¯\_(ツ)_/¯
<fake-name> is that a compile flag, or a runtime flag?
<kristina> fake-name: this is roughly how i built it except i guess no-shared is wrong: http://crna.cc/mk3wLxpiqqUyJsR
<fake-name> I should clarify, the jit is on at runtime (with precompiled binaries) by default
<kristina> also LLD's pretty colors break it.
<fake-name> if you're building it yourself, I guess it's possible to disable the jit entirely
<fake-name> how do they build it for the CI system? I'd just use that
<kristina> they don't, i'm trying to add it.
<fake-name> there's a CI system somewhere that build it, it's been complaining in-channel periodically about build failures
<cfbolz> kristina: -Ojit is what you need, yes
<cfbolz> fake-name: they aren't really *build* failures. But some tests are failing right now
<fake-name> well, the point stands that someone has automated builds
<kristina> but currently it's 1). clobbering checkout (illegal) 2). [RIGIN/../lib:/usr/local/lib] though i shouldn't need that since i can roll it into a static object if it's built as a shared library (run a semi-static linker over it but that makes it ineligible for IPO). 3). so -Ojit is what i want to produce a pypy.so that JITs objects that weren't compiled by rpython? any flags i need aside from that?
<kristina> and output would be $INSTALL_PREFIX/lib/pypy_stage1.so i guess since that's the "install" output path?
<fake-name> the troll in me really, REALLY wants to point out s/illegal/I don't agree with this/, but other then that, I have no useful knowledge
<kristina> you can do it locally, you can't commit code to the central repository that does things like that. there's a handy helper for it that injects itself into everything starting from the build system invocation (in that case shell script and hooks fs related system calls to do stuff which is either redirect/warn about misbehaving projects/fail on misbehaving projects).
<fake-name> yeah, but that's still not 'illegal', just annoying
<cfbolz> kristina: I may have missed this, but which OS/distribution are you doing this for?
<kristina> yeah but for central repository it must not do it, a single warning about traversing outside it's buildroot is enough.
<fake-name> yeah, but it's not *illegal*. Illegal implies there are laws about it
<kristina> oh this is on linux + god knows what for userland. well there are laws of the central repository like thou shall not clobber the checkout.
<fake-name> there are *standards* and hygiene, and lots of other things, but no laws
<fake-name> but arguing about that is bikeshedding
<kristina> it breaks buildbots, that's the reason it's not allowed.
<fake-name> it's self evidently allowed, since it's already in-place
<kristina> no it's on my computer.
<cfbolz> kristina: sorry, I am still missing context. What is it that you are actually trying to do?
<fake-name> from the context, set up a automated build system for an os?
<cfbolz> Yes, but which distribution? Several Linux distributions build pypy just fine
<cfbolz> kristina: hm, a thought: it's possible that running rpython with --source would be a way to tackle the problem. It should stop after generating the C source in the usession dir, so afterwards you would have to run make their, and build the cffi modules yourself
<kristina> hm, i'll say think kinda libsystem.so that has most system libraries and their exports and has most of the dynamic linker (ie. it resolves against itself). some sort of amalgamation non-gnu linux.
<fake-name> is this some weird embedded thing?
<kristina> it's for aarch64 and x86_64 pretty much, i mean i wouldn't say embedded since most of them are proper higher end-ish application processors.
<fake-name> point taken
<cfbolz> kristina: does it have a name?
<kristina> i mean it makes sense to use a single library since -latomic -ldl -lrt -pthreads (lsystemd if you're using systemd) etc, is going to be loaded in every process, you can run a single IPO pass over the entire thing.
<cfbolz> Anyway, my suggestion would be to look into --source
<kristina> yeah i was trying to but didn't find it anywhere in the docs.
<kristina> it sounds like a i need a 2 pass build? one pass to get the files to compile and 2nd pass to invoke whatever you build with?
<cfbolz> Something like that, yes. First run rpython --source -Ojit, that will give you the C source
<cfbolz> Then run make in usession
<cfbolz> Then build the cffi modules
<kristina> so the --source will not ever clobber the checkout?
<cfbolz> I am not promising anything ;-). That's really not a concern we have in mind (as you are discovering). But I think you have a decent chance that it doesn't
<kristina> i mean a lot of things like clobbering the build root which is when you just add a cmakelists.txt/build.gn for them (depending on your mood) that handles it and scrap the original build system since they do the right thing. but rpython is not that simple i guess.
jsza has joined #pypy
<kristina> cmake/gn make very few assumptions about the target that can't be overriden.
<kristina> Makefiles on the other hand well, depends on how they're written.
<fake-name> obviously the solution is to just use docker /s
<kristina> if they respect env vars 100% makefiles just work (tm).
<kristina> and i like ninjas more.
<kristina> some more complex makefile based build systems are basically fork bombs.
<kristina> which is fine but you kind of wonder why they went with this approach.
<fake-name> I can pretty much gaurantee that 99% of the build system choices out there come down to "the guy who was setting that stuff up was most familliar with/really liked <thing>"
<kristina> yeah but generally i never had issues with cmake or gn, aside from setting a few options, take rocksdb for example: http://crna.cc/VRoAX0mPtOlp4Hn
<kristina> i didn't have to edit CMakeLists.txt or use a shell script to preconfigure it.
<kristina> cmake 3.10 got OBJECT targets which is a godsend.
<cfbolz> rpython is really its own thing. More of a compiler than a build system per se
<kristina> also pretty colors break it, i'm not sure if it's a llvm bug with "is this a pty that supports escape sequences" detection or a rpython bug.
glyph has quit [Quit: End of line.]
glyph has joined #pypy
<kristina> it has like a configure stage thing where it builds those little test thingies i guess to work out what the compiler can and can't do.
<kristina> if lld or clang show a diagnostic, the escape sequences make rpython exit with something like unknown argument -<garbled character>.
<cfbolz> kristina: uh, weird
<kristina> this is either hitting a really odd corner case in llvmsupport or it's a bug in rpython.
glyph has quit [Client Quit]
<cfbolz> kristina: any simple way I can reproduce?
glyph has joined #pypy
fake-name has quit [Ping timeout: 244 seconds]
glyph_ has joined #pypy
lritter has joined #pypy
glyph has quit [Ping timeout: 245 seconds]
glyph_ is now known as glyph
<kristina> just retrying it myself without disabling colors ... seems to not happen anymore, i think that may've been a bug i fixed where it was overriding -fcolor-diagnostics in the clang's driver but it was also passed as a -Wl,--color-diagnostics (and i got rid of that too).
<kristina> so when lld was invoked, unstripped -Wl,--color-diagnostics was passed to it.
<kristina> and lld invoked by the driver doesn't do the check because, well, i guess that's also a bug but then again i doubt anyone calls ld.lld directly.
<kristina> true question is, should the driver obey the -Wl and pass the flag down or not.
<kristina> semantically it seems that -Wl, is "i really know what i'm doing just pass this down as is" since most common and cross-platform things you wouldn't need to pass explicit flags to whichever linker is invoked.
<kristina> yes i guess this is not rpython's fault, the sequences should have been stripped, and i guess build system is to blame for using that flag instead of just letting the driver handle it and strip it before it called lld.
glyph has quit [Quit: End of line.]
glyph has joined #pypy
<kristina> but yeah, in terms of the weird "libc", like the issue with musl is that for starters it's extremely poorly optimized, generally platform wise, i think more exotic arches like sh, or microblaze or m68k should maybe ship as an extra instead of mainline. glibc is even worse. musl is like really bizzare.
<kristina> on linux it intentionally clobbers some syscalls with -ENOSYS.
<kristina> some wrappers*
glyph_ has joined #pypy
<kristina> i don't know why implement the wrapper only to break it, intentionally (because musl author really thinks that it's posix or nothing).
<kristina> while oddly enough others do have wrappers.
glyph has quit [Ping timeout: 264 seconds]
<kristina> also musl author shot down the proposal for dlopen from memory.
<kristina> it's a very painful thing to do because you have to either locate and call linker's internal machinery (have to initialize TLV and register the object with the shared linker image within that task). just linking and relocating isn't enough.
agates has quit [Ping timeout: 240 seconds]
<kristina> another big issue with glibc is that for some odd reason they decided that pthread_cancel was worth force-enabling exceptions and unwind tables.
yuvipanda has quit [Ping timeout: 255 seconds]
dddddd has joined #pypy
agates has joined #pypy
yuvipanda has joined #pypy
antocuni_ has joined #pypy
antocuni_ is now known as antocuni
wleslie has joined #pypy
wleslie has quit [Quit: ~~~ Crash in JIT!]
DIRT has quit [Ping timeout: 240 seconds]
solarjoe4 has joined #pypy
antocuni has quit [Ping timeout: 240 seconds]
tayfun26 has quit [Remote host closed the connection]
mattip has joined #pypy
marky1991 has joined #pypy
tayfun26 has joined #pypy
marky1991 has quit [Read error: Connection reset by peer]
marky1991 has joined #pypy
oberstet has joined #pypy
oberstet has quit [Ping timeout: 255 seconds]
oberstet has joined #pypy
lritter has quit [Ping timeout: 265 seconds]
antocuni has joined #pypy
marvin_ has quit [Ping timeout: 256 seconds]
marvin has joined #pypy
Hasimir has quit [Ping timeout: 240 seconds]
TheAdversary has quit [Ping timeout: 264 seconds]
<kenaan> mattip unicode-utf8-py3 88b5b1b85e78 /pypy/: utf_8 -> utf8_w, str_w -> text_w
<kenaan> mattip unicode-utf8-py3 2621c0f70d91 /pypy/: start to pass unicode around, try using decode_utf8sp
<kenaan> mattip unicode-utf8-py3 4ef833b2310d /pypy/module/_sre/interp_sre.py: revert interp_sre.py to py3.5
<kenaan> mattip unicode-utf8-py3 388bbf987266 /pypy/module/: fixes for py3.5 _sre module
<bbot2> Started: http://buildbot.pypy.org/builders/own-linux-x86-64/builds/6852 [mattip: force build, unicode-utf8-py3]
adamholmberg has joined #pypy
marky1991 has quit [Ping timeout: 268 seconds]
Hasimir has joined #pypy
TheAdversary has joined #pypy
stevenja_ has joined #pypy
bananabread has quit [Ping timeout: 255 seconds]
Rhy0lite has joined #pypy
marky1991 has joined #pypy
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
Graypup_ has quit [Quit: ZNC 1.6.1 - http://znc.in]
Graypup_ has joined #pypy
marky1991 has quit [Ping timeout: 244 seconds]
marky1991 has joined #pypy
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
<bbot2> Failure: http://buildbot.pypy.org/builders/own-linux-x86-64/builds/6852 [mattip: force build, unicode-utf8-py3]
lesshaste has joined #pypy
solarjoe4 has quit [Quit: Leaving]
stevenja_ has quit [Remote host closed the connection]
stevenja_ has joined #pypy
stevenja_ has quit [Ping timeout: 244 seconds]
stevenja_ has joined #pypy
oberstet has quit [Ping timeout: 255 seconds]
stevenja_ has quit [Remote host closed the connection]
stevenja_ has joined #pypy
altendky has joined #pypy
stevenja_ has quit [Remote host closed the connection]
marky1991 has quit [Ping timeout: 244 seconds]
stevenja_ has joined #pypy
oberstet has joined #pypy
tayfun26 has quit [Remote host closed the connection]
stevenja_ has quit [Remote host closed the connection]
stevenja_ has joined #pypy
stevenja_ has quit [Remote host closed the connection]
TheAdversary has quit [Ping timeout: 240 seconds]
stevenja_ has joined #pypy
Hasimir has quit [Ping timeout: 240 seconds]
lritter has joined #pypy
marky1991 has joined #pypy
mattip has quit [Ping timeout: 256 seconds]
Hasimir has joined #pypy
oberstet has quit [Ping timeout: 244 seconds]
TheAdversary has joined #pypy
mattip has joined #pypy
Hasimir has quit [Read error: Connection reset by peer]
TheAdversary has quit [Read error: Connection reset by peer]
TheAdversary has joined #pypy
Hasimir has joined #pypy
antocuni has quit [Ping timeout: 264 seconds]
stevenja_ has quit [Remote host closed the connection]
stevenja_ has joined #pypy
stevenja_ has quit [Ping timeout: 244 seconds]
stevenja_ has joined #pypy
stevenja_ has quit [Ping timeout: 240 seconds]
glyph_ is now known as glyph
stevenja_ has joined #pypy
stevenja_ has quit [Remote host closed the connection]
stevenja_ has joined #pypy
oberstet has joined #pypy
DIRT has joined #pypy
stevenja_ has quit [Remote host closed the connection]
jamesaxl has joined #pypy
stevenja_ has joined #pypy
stevenja_ has quit [Remote host closed the connection]
stevenja_ has joined #pypy
stevenja_ has quit [Ping timeout: 260 seconds]
marky1991 has quit [Read error: Connection reset by peer]
marky1991 has joined #pypy
Rhy0lite has quit [Quit: Leaving]
marky1991 has quit [Ping timeout: 244 seconds]
marky1991 has joined #pypy
stevenja_ has joined #pypy
mattip has quit [Ping timeout: 276 seconds]
mattip has joined #pypy
marky1991 has quit [Ping timeout: 276 seconds]
marky1991 has joined #pypy
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
antocuni has joined #pypy
stevenja_ has quit [Remote host closed the connection]
stevenja_ has joined #pypy
stevenja_ has quit [Ping timeout: 260 seconds]
marky1991 has quit [Ping timeout: 240 seconds]
oberstet has quit [Quit: Leaving]
marky1991 has joined #pypy
stevenja_ has joined #pypy
stevenja_ has quit [Ping timeout: 264 seconds]
jamesaxl has quit [Quit: WeeChat 2.1]
lesshaste has quit [Ping timeout: 276 seconds]
lesshaste has joined #pypy
lesshaste is now known as Guest42320
antocuni has quit [Ping timeout: 264 seconds]
adamholm_ has joined #pypy
adamholmberg has quit [Read error: Connection reset by peer]
blachance has quit [Ping timeout: 240 seconds]
blachance has joined #pypy
energizer has quit [Ping timeout: 260 seconds]
__peke__ has quit [Ping timeout: 260 seconds]
adamholm_ has quit [Ping timeout: 240 seconds]
energizer has joined #pypy
__peke__ has joined #pypy
lritter has quit [Ping timeout: 240 seconds]
wleslie has joined #pypy
adamholmberg has joined #pypy