cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | the secret reason for us trying to get PyPy users: to test the JIT well enough that we're somewhat confident about it
altendky has quit [Quit: Connection closed for inactivity]
realitix has quit [Ping timeout: 260 seconds]
<trfl>
I have noticed a difference when 1) letting `rpython [...] -Ojit targetpypystandalone` handle the entire build process, and 2) when adding --source and then calling make manually in /tmp/usession-*-$(whoami)/testing_1
<LarstiQ>
trfl: namely?
<trfl>
the build takes about the same amount of time however libpypy-c.so.debug ends up being ~18'270'136 byte with option 1, and ~1'394'496'520 byte with option 2
<trfl>
I haven't really had a look at the actual difference in contents yet, just figured there must be some step i'm missing - maybe stripping useless symbols or something
<arigato>
what is "libpypy-c.so.debug"?
DIRT has quit [Remote host closed the connection]
<trfl>
oh sorry, forgot -- it's an implementation detail in the packaging system that separates out symbol information from the .so that is generated
<trfl>
after that separation the library itself turns out roughly the same size, originally straight out of the `make` call the generated .so is 1'465'995'080
antocuni has joined #pypy
<trfl>
looks like my assumption was partially incorrect, the first 90% of the file is all non-textual data, the final 10% being 408'261 symbols (2'467'629 strings) of the form l_v%5d and l_obj_%4d
<arigato>
I cannot help, I've never seen .so's so incredibly huge
<trfl>
for the record this is what my build steps look like, just in case I'm missing something obvious (as I was hoping) https://pastebin.com/h7yfMK5A
<arigato>
"strip" on the two versions reduces them to the same size, but I have no clue what's the difference
<trfl>
maybe the make environment as set up by rpython doesn't include as much debug information in the .so? I ended up in rpython/translator/platform/posix.py and rpython/translator/c/genc.py but couldn't spot anything right away
<arigato>
rpython also just calls "make"...
<arigato>
ah, I see "-g0" somewhere
realitix has joined #pypy
<arigato>
ok, no clue any more: "make" produces the same 86MB file for me
dddddd has joined #pypy
<arigato>
it seems I was confused, and maybe you too---are you sure the 1.4GB version wasn't built with "make lldebug" or some other option?
<arigato>
note that you need "make clean" if you plan to recompile sources with another option
<arigato>
how big is e.g. implement_10.o? it is 700KB if built in release mode, and 6-7MB if built in debug mode
<trfl>
I'm sure that no arguments were provided to the make call, however the build framework we're using might have set some environment variables that affects the process... i'm doing a plain old make right now, shouldn't take too long
<arigato>
(sorry, 9MB)
<arigato>
you can try "make implement_10.o" and look at the same
<trfl>
comes out as 9'661'704, so I guess our make defaults to building debug somehow
<arigato>
ah
<arigato>
do you have a $CFLAGS?
<arigato>
...no, shouldn't matter, but can you check the command-line: it should say "gcc -O3 -pthread ..etc.." with no "-g"
<trfl>
nothing in /proc/$(pidof make)/environ at least, however in the gcc args I can see -D_REENTRANT -D Py_BUILD_CORE -D PYPY_JIT_CODEMAP -D Py_BUILD_CORE -D RPYTHON_VMPROF -D VMPROF_UNIX -D VMPROF_LINUX -D RPYTHON_VMPROF=1 -D Py_BUILD_CORE -D PYPY_CPU_HAS_STANDARD_PRECISION -D RPYTHON_VMPROF=1 -D RPYTHON_VMPROF=1 -D Py_BUILD_CORE pypy_module_cStringIO.c -quiet -dumpbase pypy_module_cStringIO.c -mtune=generic -march=x86-64 -auxbase-strip
<arigato>
so either $CFLAGS or $CFLAGSEXTRA, or the Makefile was tempered with
<arigato>
no, shouldn't be $CFLAGSEXTRA
<trfl>
are $CFLAGS or $CFLAGSEXTRA considered by rpython as the makefile is created in the first place? meanwhile I'll check whether we set anything custom at that point
<arigato>
I don't remember, need to check
<arigato>
there are three extra options in the middle of CFLAGS
<arigato>
can you check in the Makefile if these extra options (including "-g") are in the multiline "CFLAGS = ..." construct?
<arigato>
...yes, you're right
<arigato>
for some reason we include the current $CFLAGS inside the Makefile when we produce it
<trfl>
we set CFLAGS="-Os -fomit-frame-pointer" for sure, haven't checked what it actually ends up being at compile time yet -- but in the makefile I can see CFLAGS = [...] -Wno-address -Os -fomit-frame-pointer -g -fPIC [...]
<arigato>
ok
<trfl>
man, looks like that's it! at the time of running rpython, CXXFLAGS=-Os -fomit-frame-pointer -g
<arigato>
CXXFLAGS only? this should not be used
<trfl>
oh wait, CFLAGS and CPPFLAGS too, sorry
<arigato>
ok, so maybe a guess would be that you run "rpython" and "rpython --source" in environments which differ in $CFLAGS containing "-g" or not?
<arigato>
(and the environment's $CFLAGS is not used where you type "make")
<trfl>
actually no, the same environment varable was set for sure when letting rpython handle it without --source
<trfl>
so that's the spooky part :p
<arigato>
note that "strip" is run by pypy/tool/release/package.py, too, but this shouldn't execute automatically if you just do "rpython"
<trfl>
oh then that would explain it, I do run package.py as soon as the build completes -- I'm presuming that modifies the .so in /tmp/usession-* directly?
<trfl>
s/directly/in-place/
<arigato>
no, it shouldn't
<arigato>
but it makes a new /tmp/usession-*
<arigato>
directory
<arigato>
libpypy-c.so is copied, I think, from pypy/goal/libpypy-c.so, by default
<arigato>
but you can add parameters
<arigato>
and maybe "strip" is applied on it, so by default on pypy/goal/libpypy-c.so
<arigato>
but it could be anywhere else if you added parameters
<trfl>
alright, well that explains all of it regardless - guess I just never noticed the initial size of the library heh
<arigato>
yes, as you noticed, I was also surprized myself :-)
<trfl>
thanks a bunch for your help, especially since it wasn't really a pypy-thing after all!
<arigato>
NP
<kenaan_>
arigo default 0ee5333ce97e /: rvmprof.dummy is not really working, at least for pypy. Try to fix it some more, but give up for now. Instead, ro...
antocuni has quit [Ping timeout: 264 seconds]
antocuni has joined #pypy
Rhy0lite has joined #pypy
marky1991 has joined #pypy
stevenja_ has joined #pypy
adamholmberg has joined #pypy
marky1991 has quit [Read error: Connection reset by peer]
stevenja_ has quit [Remote host closed the connection]
realitix_ has joined #pypy
stevenja_ has joined #pypy
realitix has quit [Ping timeout: 244 seconds]
realitix_ is now known as realitix
stevenja_ has quit [Remote host closed the connection]
stevenja_ has joined #pypy
stevenja_ has quit [Ping timeout: 245 seconds]
stevenja_ has joined #pypy
realitix has quit [Quit: realitix]
realitix has joined #pypy
stevenja_ has quit [Remote host closed the connection]
stevenja_ has joined #pypy
mattip has quit [Ping timeout: 244 seconds]
mattip has joined #pypy
stevenja_ has quit [Remote host closed the connection]
<kenaan_>
arigo py3.5 cad71ec84cef /lib_pypy/_winapi.py: Start to debug 'multiprocessing' on win32, but found out that '_winapi.py' is not complete at all.
stevenja_ has joined #pypy
stevenja_ has quit [Remote host closed the connection]
stevenja_ has joined #pypy
stevenja_ has quit [Ping timeout: 240 seconds]
stevenja_ has joined #pypy
stevenja_ has quit [Remote host closed the connection]
stevenja_ has joined #pypy
rubdos has quit [Ping timeout: 256 seconds]
realitix has quit [Quit: realitix]
realitix has joined #pypy
realitix has quit [Client Quit]
realitix has joined #pypy
rubdos has joined #pypy
stevenja_ has quit [Remote host closed the connection]
stevenja_ has joined #pypy
stevenja_ has quit [Remote host closed the connection]