cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | "the modern world where network packets and compiler optimizations are effectively hostile"
jcea has quit [Ping timeout: 276 seconds]
jcea has joined #pypy
marr has quit [Ping timeout: 240 seconds]
marvin_ has quit [Ping timeout: 240 seconds]
tilgovi has quit [Remote host closed the connection]
tilgovi has joined #pypy
yuyichao_ has quit [Read error: No route to host]
yuyichao has joined #pypy
jcea has quit [Remote host closed the connection]
marvin_ has joined #pypy
kipras is now known as kipras`away
forgottenone has joined #pypy
exarkun has quit [Ping timeout: 260 seconds]
exarkun has joined #pypy
https_GK1wmSU has joined #pypy
https_GK1wmSU has left #pypy [#pypy]
Ryanar has joined #pypy
ssbr has quit [Ping timeout: 246 seconds]
ssbr has joined #pypy
forgottenone has quit [Quit: Konversation terminated!]
tilgovi has quit [Ping timeout: 255 seconds]
slackyy has quit [Ping timeout: 246 seconds]
<kenaan> rlamy cpyext-leakchecking 93ef9a8f432f /pypy/interpreter/baseobjspace.py: Add space.appdef()
<kenaan> mattip cpyext-leakchecking c1d2c7124ad5 /rpython/rtyper/: fix translation by adding immortal to malloc in rpython (grafted from e3219c61e7a801f5257cba1a9df98aa1...
<kenaan> rlamy cpyext-leakchecking d7afdd455feb /pypy/interpreter/baseobjspace.py: Add option to prevent caching in space.appexec() (for cpyext tests)
<kenaan> rlamy cpyext-leakchecking d893df676f94 /pypy/module/cpyext/test/test_dictobject.py: Move 2 tests from app- to interp-level to fix refcounting issues
ArneBab_ has joined #pypy
Ryanar has quit [Quit: Ryanar]
ArneBab has quit [Ping timeout: 260 seconds]
exarkun has quit [Ping timeout: 240 seconds]
exarkun has joined #pypy
<kenaan> rlamy cpyext-leakchecking 398723725f36 /pypy/module/cpyext/test/: fix some tests
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
toolchains_x86_ has quit [Ping timeout: 260 seconds]
tilgovi has joined #pypy
tilgovi has quit [Remote host closed the connection]
raynold has quit [Quit: Connection closed for inactivity]
exarkun has quit [Ping timeout: 240 seconds]
exarkun has joined #pypy
forgottenone has joined #pypy
yuyichao has quit [Ping timeout: 255 seconds]
yuyichao has joined #pypy
yuyichao has quit [Read error: Connection reset by peer]
yuyichao_ has joined #pypy
forgottenone has quit [Quit: Konversation terminated!]
zware has quit [Ping timeout: 240 seconds]
zware has joined #pypy
exarkun has quit [Ping timeout: 246 seconds]
exarkun has joined #pypy
dmalcolm has quit [Ping timeout: 268 seconds]
<kenaan> cfbolz optinfo-into-bridges 596f167b7fd8 /: close abandoned branch (everything useful was merged via the optinfo-into-bridges-3 branch)
Remi_M has joined #pypy
dmalcolm has joined #pypy
runciter has quit [Ping timeout: 260 seconds]
runciter has joined #pypy
https_GK1wmSU has joined #pypy
https_GK1wmSU has left #pypy [#pypy]
yuyichao_ has quit [Remote host closed the connection]
oberstet has joined #pypy
exarkun has quit [Ping timeout: 246 seconds]
exarkun has joined #pypy
vkirilichev has joined #pypy
https_GK1wmSU has joined #pypy
https_GK1wmSU has left #pypy [#pypy]
antocuni has joined #pypy
gumblex has joined #pypy
gumblex_ has quit [Ping timeout: 255 seconds]
slackyy has joined #pypy
marr has joined #pypy
exarkun has quit [Ping timeout: 255 seconds]
exarkun has joined #pypy
<kenaan> cfbolz arrays-force-less da044fcdd451 /: a branch to try to not force the whole array after a lazy setarrayitem is forced, but only the affected ...
<kenaan> cfbolz arrays-force-less 7a124bb988d9 /rpython/jit/metainterp/optimizeopt/: a small improvement to getarrayitem caching when invalidating getarrayitem caches due to a setarrayitem...
<bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/4799 [Carl Friedrich Bolz-Tereick: force build, arrays-force-less]
<bbot2> Started: http://buildbot.pypy.org/builders/own-linux-x86-64/builds/6112 [Carl Friedrich Bolz-Tereick: force build, arrays-force-less]
antocuni has quit [Ping timeout: 260 seconds]
* arigato writes examples in the cffi tests where CPython's cyclic GC suffers from the same problem as PyPy's GC: unexpectedly large objects, causing the GC to think everything's fine when it's not
<arigato> it seems that __pypy__.add_memory_pressure() should really exist on CPython, too
mihneas has joined #pypy
mihneas has quit [Client Quit]
<LarstiQ> how would it work with cpython's gc?
<LarstiQ> __pypy__ calling gc.collect?
<arigato> unsure
<cfbolz> arigato: argh
bgola_ is now known as bgola
lritter has joined #pypy
<fijal> arigato: that sounds like a major bikeshedding topic :)
* fijal is stuck trying to deal with banks
nimaje1 has joined #pypy
nimaje1 is now known as nimaje
nimaje has quit [Killed (verne.freenode.net (Nickname regained by services))]
<fijal> cpython optimizatins discussions
<arigato> fijal: of course, I'm thinking along the lines of: add ffi.add_memory_pressure() to cffi
<fijal> of course
<fijal> you can add cpython hack
<fijal> is it worth mentioning or not really?
exarkun has quit [Ping timeout: 255 seconds]
<arigato> ?
<fijal> to cpython-dev
<arigato> no
exarkun has joined #pypy
<fijal> :]
<arigato> unless they care about the same example with ctypes
<arigato> of course the same example works with about any C extension module
<fijal> of course
<fijal> does it work with numpy?
<arigato> yes, certainly
<fijal> maybe it's worth mentioning then?
<fijal> although I'm sure there is a "disable the cyclic GC" camp
<arigato> no, if you disable the cyclic GC you'll just leak these objects
* arigato trying to understand CPython's GC thresholds
<fijal> arigato: sure, but then you can inspect stuff by hand in gc.garbage and not create cycles, see?
<arigato> right
jcea has joined #pypy
<bbot2> Success: http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/4799 [Carl Friedrich Bolz-Tereick: force build, arrays-force-less]
<arigato> uh
<arigato> ...right, but you need some other debugging functions in the 'gc' module than just disabling cyclic GC
<arigato> seems impossible to do without major hacks (or changes to CPython's source code, of course)
<arigato> the logic is just couning objects, and that's it
<arigato> counting
<arigato> you can already badly overflow that logic with code like that (will exhaust my 8GB of RAM very quickly):
<bbot2> Failure: http://buildbot.pypy.org/builders/own-linux-x86-64/builds/6112 [Carl Friedrich Bolz-Tereick: force build, arrays-force-less]
<arigato> bug report? :-/
<arigato> the cyclic GC will run every 700 allocated objects
<arigato> so the same example exhausts all memory as long as the list's size is one-700th of that memory
antocuni_ has joined #pypy
<kenaan> exarkun py3.5 4499d3bd3e6f /pypy/objspace/std/unicodeobject.py: Fix test_unflagged_non_text_codec_handling test_unflagged_non_text_codec_handling (test.test_codecs.ExceptionChain...
jkbbwr has joined #pypy
antocuni_ is now known as antocuni
Ryanar has joined #pypy
yuyichao has joined #pypy
Rhy0lite has joined #pypy
<kenaan> arigo cffi/cffi 120347b84c08 /: Add an optional 'size' argument to ffi.gc(). So far, it has no effect on CPython.
marky1991 has quit [Ping timeout: 260 seconds]
<kenaan> antocuni default 620fefe235f3 /lib-python/2.7/distutils/unixccompiler.py: Add a workaround to make the distutils options 'runtime_library_dirs' working. Fixes #2593
adamholmberg has joined #pypy
<kenaan> arigo default 53cb371f1828 /pypy/module/_cffi_backend/: Call add_memory_pressure(size) in response to ffi.gc(..., size)
<kenaan> arigo default 1903d256e24b /: import cffi/120347b84c08
<kenaan> arigo default e4d42cef80b0 /lib-python/2.7/distutils/unixccompiler.py: merge heads
make has joined #pypy
<fijal> arigato: you sure it did not get (somehow) fixed on py3k?
<arigato> fijal: I checked in CPython 3.7
<make> Hey! :) I'm trying to build PyPy from the source. My question would be, would running make in the directory and following these instructions do the same thing? http://doc.pypy.org/en/latest/build.html
<make> I did the former, but I am not sure if I made the pypy-c executable.
<LarstiQ> make: running make is fine
<make> Is there anything else I need to do after running make then? I still get an error when trying to use PyPy
<fijal> ok
marky1991 has joined #pypy
<fijal> hm, I thought they fixed the quadratic behavior
<LarstiQ> make: what error?
<kenaan> arigo cffi/cffi 5b1214633c90 /doc/source/: Document ffi.gc(..., size)
<make> LarstiQ: pypy: not found
<arigato> fijal: talking about the GC in CPython? yes, this changed
<arigato> it has nothing to do with the problem I described here, though
<arigato> or at least their solution has nothing to do, because that solution is still based on the number of objects, not the size that each object keeps around
<fijal> ah
<make> I tried to package the version I ran make on but I keep getting this error: Please compile pypy first, using translate.py, or check that you gave the correct path with --override_pypy_c
lritter has quit [Quit: Leaving]
<LarstiQ> make: doing what? It sounds like you just typed 'pypy' which won't do anything
<LarstiQ> you need to run the pypy-c binary it produced
<make> LarstiQ: how do I run it?
<arigato> I think this error means that "make" didn't succeed
<arigato> otherwise, there is a "pypy/goal/pypy-c" binary
<arigato> ah, or maybe a "./pypy-c" binary, not inside "pypy/goal"
<arigato> and, re "ah", it might be because of this different location that you get the error
<kenaan> cfbolz arrays-force-less 5d7f91496e04 /pypy/doc/whatsnew-head.rst: document branch
<kenaan> cfbolz arrays-force-less 82b3076c7887 /: close to-be-merged branch
<arigato> anyone has any objection if I change the Makefile inside pypy to produce "pypy/goal/pypy-c", as done by all other documented ways and as expected by all other places?
<cfbolz> arigato: yes please
<arigato> we neither test, not really document, the Makefile
<kenaan> cfbolz default a22d0969f40a /: merge arrays-force-less don't throw away all of the knowledge about an array when forcing a lazy setarrayitem. in...
<make> arigato: This might sound like a stupid question, but after packaging, how do I install and run? I have tried everything, but it seems like I'm not doing the correct thing.
<arigato> unpack the .tar.bz2 file somewhere, and run "bin/pypy" there
<arigato> we recommend /opt/pypy5.8/ as a place, for example
marky1991 has quit [Read error: Connection reset by peer]
marky1991 has joined #pypy
<kenaan> cfbolz getarrayitem-into-bridges 046804fa21b3 /rpython/jit/metainterp/: in progress
<kenaan> cfbolz getarrayitem-into-bridges e8af759e759d /rpython/jit/metainterp/optimizeopt/: merge arrays-force-less
<kenaan> cfbolz getarrayitem-into-bridges 8885e06c04d6 /rpython/jit/metainterp/: finish support for serializing knowledge about arrays
<kenaan> cfbolz getarrayitem-into-bridges a9a9281dad98 /rpython/jit/metainterp/: fix affected tests
<kenaan> cfbolz getarrayitem-into-bridges d16b10f8ca91 /: merge default
<bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/4800 [Carl Friedrich Bolz-Tereick: force build, getarrayitem-into-bridges]
<bbot2> Started: http://buildbot.pypy.org/builders/own-linux-x86-64/builds/6113 [Carl Friedrich Bolz-Tereick: force build, getarrayitem-into-bridges]
<bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-32/builds/4109 [Carl Friedrich Bolz-Tereick: force build, arrays-force-less]
make has quit [Ping timeout: 260 seconds]
vkirilichev has quit [Remote host closed the connection]
vkirilichev has joined #pypy
vkirilichev has quit [Ping timeout: 240 seconds]
tbodt has joined #pypy
Ryanar has quit [Quit: Ryanar]
exarkun has quit [Read error: Connection reset by peer]
<bbot2> Started: http://buildbot.pypy.org/builders/own-linux-x86-32/builds/5348 [Carl Friedrich Bolz-Tereick: force build, arrays-force-less]
exarkun has joined #pypy
forgottenone has joined #pypy
<kenaan> arigo default dd95b2eeb5ac /Makefile: Fix the Makefile to produce the pypy-c inside pypy/goal/, like we normally expect and document
mvantellingen has quit [Ping timeout: 240 seconds]
oberstet has quit [Ping timeout: 260 seconds]
raynold has joined #pypy
* ronan is trying to use revdb again
<ronan> arigato: do you have any idea how I could set a watch on a c_ob_refcnt?
<bbot2> Failure: http://buildbot.pypy.org/builders/own-linux-x86-64/builds/6113 [Carl Friedrich Bolz-Tereick: force build, getarrayitem-into-bridges]
mattip has joined #pypy
<mattip> ronan: hi. in gdb, you need to get the address of the PyObject
<mattip> then it's something like "watch ((PyObject*)0xffaaddeedd)->ob_refcnt
<mattip> "
<ronan> mattip: yes, but that can't work in revdb
<ronan> it'll complain about "Attempted to do I/O or access raw memory"
<bbot2> Success: http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/4800 [Carl Friedrich Bolz-Tereick: force build, getarrayitem-into-bridges]
<mattip> no idea then, sorry
<ronan> mattip: BTW, refcount management of memoryviews/buffers seems to be still wrong
<mattip> :(
<ronan> mattip: in test_frombuffer(), w_memoryview should be decref'd, but then things break
oberstet has joined #pypy
<mattip> I'm staring at the test_typeobject test_tp_call, which leaks, here is acopy of the test
<mattip> line 20 leaks, line 26 does not
<mattip> so we do not properly handle user-defined classes ?
* mattip off to a movie
mattip has left #pypy ["bye"]
jcea has quit [Quit: jcea]
<antocuni> ronan: I think that revdb works by using an rpython transformation which logs all the read/writes to known objects. If it's raw memory, it cannot possibly keep track of it
<ronan> antocuni: yes, but here all the writes are done from Python code, so I'm hoping there's a way to simulate watching it
<antocuni> no clue, sorry :(
mattip has joined #pypy
<kenaan> mattip cpyext-leakchecking c1786ec814d9 /pypy/module/cpyext/typeobject.py: maybe identify a possible leak in handling GetSetProperty?
<mattip> ronan: that commit has nothing to do with user-defined classes, but GetSet has also been troubling me
jacob22_ has quit [Quit: Konversation terminated!]
<mattip> now I am really away
mattip has left #pypy ["bye"]
<ronan> mattip (logs): AFAICT, it's never released
<kenaan> cfbolz getarrayitem-into-bridges f105d8c2eae7 /rpython/jit/metainterp/: support for caching infos on constants, while I'm at it
lritter has joined #pypy
lritter has quit [Remote host closed the connection]
lritter has joined #pypy
antocuni has quit [Ping timeout: 240 seconds]
yuyichao_ has joined #pypy
yuyichao has quit [Ping timeout: 255 seconds]
yuyichao_ has quit [Ping timeout: 246 seconds]
kanaka has quit [Remote host closed the connection]
yuyichao_ has joined #pypy
slackyy has quit [Ping timeout: 240 seconds]
lritter has quit [Quit: Leaving]
mvantellingen has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
kipras`away is now known as kipras
tbodt has joined #pypy
jacob22_ has joined #pypy
Rhy0lite has quit [Quit: Leaving]
https_GK1wmSU has joined #pypy
https_GK1wmSU has left #pypy [#pypy]
vkirilichev has joined #pypy
Rhy0lite has joined #pypy
mvantellingen has quit [Ping timeout: 240 seconds]
adamholmberg has quit [Remote host closed the connection]
mvantellingen has joined #pypy
yuyichao_ has quit [Ping timeout: 240 seconds]
yuyichao_ has joined #pypy
vkirilichev has quit [Remote host closed the connection]
marky1991 has quit [Ping timeout: 255 seconds]
https_GK1wmSU has joined #pypy
https_GK1wmSU has left #pypy [#pypy]
slackyy has joined #pypy
<bbot2> Success: http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-32/builds/4109 [Carl Friedrich Bolz-Tereick: force build, arrays-force-less]
<bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-32/builds/4110 [Carl Friedrich Bolz-Tereick: force build, getarrayitem-into-bridges]
<bbot2> Success: http://buildbot.pypy.org/builders/own-linux-x86-32/builds/5348 [Carl Friedrich Bolz-Tereick: force build, arrays-force-less]
<bbot2> Started: http://buildbot.pypy.org/builders/own-linux-x86-32/builds/5349 [Carl Friedrich Bolz-Tereick: force build, getarrayitem-into-bridges]
<arigato> ronan: when replaying there is no c_ob_refcnt at all
<arigato> as in, the whole PyObject is not allocated
<arigato> you can put breakpoints on Py_IncRef() and friends if you're sure you don't forget one, and do the counting manually
<arigato> ...except you can't put breakpoints at this level, of course
<arigato> I guess I'd recommend using gdb, not revdb
<arigato> and possibly "rr" if you need gdb + record/replay
* arigato zz
yuyichao_ has quit [Quit: Konversation terminated!]
yuyichao_ has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
pilne has joined #pypy
antocuni has joined #pypy
yuyichao_ has quit [Ping timeout: 248 seconds]