#pypy on 2018-06-09 — irc logs at freenode.irclog.whitequark.org

2018-02-26 15:52 cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | the secret reason for us trying to get PyPy users: to test the JIT well enough that we're somewhat confident about it

00:12 ceridwen has quit [Ping timeout: 245 seconds]

00:29 ceridwen has joined #pypy

00:29 ceridwen has quit [Changing host]

00:44 _whitelogger has joined #pypy

00:51 adamholmberg has joined #pypy

00:52 adamholmberg has quit [Remote host closed the connection]

00:52 adamholmberg has joined #pypy

00:57 adamholmberg has quit [Ping timeout: 276 seconds]

01:00 inhahe__ has joined #pypy

01:01 <bbot2> Started: http://buildbot.pypy.org/builders/build-pypy-c-jit-linux-armel/builds/2317 [py3.5]

01:01 <bbot2> Started: http://buildbot.pypy.org/builders/build-pypy-c-jit-linux-armhf-raspbian/builds/2078 [py3.5]

01:02 <bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/5525 [py3.5]

01:02 <bbot2> Started: http://buildbot.pypy.org/builders/own-linux-x86-64/builds/6803 [py3.5]

01:02 <bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/3957 [py3.5]

01:02 <bbot2> Started: http://buildbot.pypy.org/builders/own-linux-x86-32/builds/5907 [py3.5]

01:02 <bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-32/builds/4673 [py3.5]

01:39 <bbot2> Failure: http://buildbot.pypy.org/builders/pypy-c-jit-linux-armhf-raspbian/builds/1767 [py3.5]

01:39 jcea has quit [Quit: jcea]

01:52 adamholmberg has joined #pypy

02:10 bbot2 has quit [Ping timeout: 256 seconds]

02:19 ceridwen has quit [Ping timeout: 265 seconds]

02:31 ceridwen has joined #pypy

02:31 bbot2 has joined #pypy

02:31 <bbot2> Failure: http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/5525 [py3.5]

02:36 <bbot2> Success: http://buildbot.pypy.org/builders/build-pypy-c-jit-linux-armhf-raspbian/builds/2078 [py3.5]

02:37 <bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-jit-linux-armhf-raspbian/builds/1768 [py3.5]

02:44 <bbot2> Failure: http://buildbot.pypy.org/builders/own-linux-x86-32/builds/5907 [py3.5]

02:47 ceridwen has quit [Ping timeout: 265 seconds]

02:53 ceridwen has joined #pypy

03:00 ceridwen has quit [Ping timeout: 260 seconds]

03:01 tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]

03:07 ceridwen has joined #pypy

03:22 ceridwen has quit [Ping timeout: 276 seconds]

03:24 ceridwen has joined #pypy

03:39 ceridwen has quit [Ping timeout: 260 seconds]

03:45 ceridwen has joined #pypy

03:51 ceridwen has quit [Ping timeout: 276 seconds]

03:54 ceridwen has joined #pypy

04:06 ceridwen has quit [Ping timeout: 276 seconds]

04:11 ceridwen has joined #pypy

04:18 ceridwen has quit [Ping timeout: 276 seconds]

04:24 ceridwen has joined #pypy

04:41 _whitelogger has joined #pypy

04:41 <kenaan> arigo default 937d78fe0a37 /rpython/rlib/rposix_environ.py: Issue #2840 Fix multithreading issues in calls to setenv()

04:43 jamesaxl has quit [Quit: WeeChat 2.1]

04:45 bbot2 has quit [Ping timeout: 264 seconds]

04:48 <arigato> plan_rich (for the logs): you have a runaway twistd process on baroquesoftware.com, gobbling all the memory. I killed it

04:56 bbot2 has joined #pypy

04:57 <bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-jit-linux-armhf-raspbian/builds/1769 [py3.5]

04:58 <bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/3958 [py3.5]

04:58 <bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-jit-linux-armhf-raspbian/builds/1770 [py3.5]

04:59 <bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-jit-macosx-x86-64/builds/3823 [py3.5]

05:07 ceridwen has quit [Ping timeout: 276 seconds]

05:12 ceridwen has joined #pypy

05:31 hotpot33 has quit [Ping timeout: 264 seconds]

05:31 ceridwen has quit [Ping timeout: 245 seconds]

05:32 hotpot33 has joined #pypy

05:34 ceridwen has joined #pypy

05:45 <kenaan> wlav cppyy-packaging 068d5604f6e9 /pypy/module/_cppyy/interp_cppyy.py: method template improvements

05:45 <kenaan> wlav cppyy-packaging c68cd6b1c308 /pypy/module/_cppyy/: further support for templated methods and for sfinae

06:02 <bbot2> Failure: http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-32/builds/4674 [py3.5]

06:06 hotpot33 has quit [Ping timeout: 256 seconds]

06:08 oberstet has joined #pypy

06:08 <kenaan> mattip default d246497b019e /pypy/module/cpyext/include/pyconfig.h: add missing defines and typedefs for numpy, pandas on MSVC

06:08 <kenaan> mattip default 109e417f3057 /pypy/doc/windows.rst: try to describe setting up a current MSVC build env for py3, until it changes

06:11 <kenaan> mattip py3.5 2e791fab4895 /: merge default into branch

06:46 ceridwen has quit [Ping timeout: 260 seconds]

06:53 ceridwen has joined #pypy

07:19 ceridwen has quit [Ping timeout: 276 seconds]

07:23 ceridwen has joined #pypy

07:46 ceridwen has quit [Ping timeout: 260 seconds]

07:51 ceridwen has joined #pypy

08:43 <bbot2> Failure: http://buildbot.pypy.org/builders/pypy-c-jit-macosx-x86-64/builds/3823 [py3.5]

09:10 energizer is now known as energizer_

09:11 energizer_ is now known as energizer

09:46 hotpot33 has joined #pypy

09:51 hotpot33 has quit [Ping timeout: 245 seconds]

09:55 hotpot33 has joined #pypy

09:59 ceridwen has quit [Ping timeout: 260 seconds]

10:01 ceridwen has joined #pypy

10:25 exarkun has quit [Read error: Connection reset by peer]

10:34 exarkun has joined #pypy

10:43 ceridwen has quit [Ping timeout: 260 seconds]

10:48 ceridwen has joined #pypy

11:00 inad922 has joined #pypy

11:09 hotpot33 has quit [Read error: Connection reset by peer]

11:22 bremner has quit [Quit: Coyote finally caught me]

11:23 bremner has joined #pypy

11:23 inad922 has quit [Quit: Leaving]

11:23 Hasimir has quit [Ping timeout: 240 seconds]

11:23 bremner has quit [Client Quit]

11:24 Hasimir has joined #pypy

11:57 dddddd has joined #pypy

11:58 jcea has joined #pypy

12:15 antocuni has joined #pypy

13:33 bremner has joined #pypy

13:37 adamholmberg has quit [Remote host closed the connection]

13:37 adamholmberg has joined #pypy

13:37 adamholmberg has quit [Remote host closed the connection]

13:40 ceridwen has quit [Ping timeout: 265 seconds]

13:43 ceridwen has joined #pypy

13:49 Taggnostr3 has joined #pypy

13:50 Taggnostr2 has quit [Ping timeout: 248 seconds]

13:53 Taggnostr2 has joined #pypy

13:54 Taggnostr3 has quit [Ping timeout: 260 seconds]

13:54 ceridwen has quit [Ping timeout: 265 seconds]

13:57 adamholmberg has joined #pypy

13:58 Taggnostr3 has joined #pypy

13:59 Taggnostr2 has quit [Ping timeout: 268 seconds]

14:01 adamholmberg has quit [Ping timeout: 240 seconds]

14:07 ceridwen has joined #pypy

14:22 Taggnostr2 has joined #pypy

14:23 Taggnostr3 has quit [Ping timeout: 265 seconds]

14:25 ceridwen has quit [Ping timeout: 276 seconds]

14:27 ceridwen has joined #pypy

14:29 Taggnostr3 has joined #pypy

14:30 Taggnostr2 has quit [Ping timeout: 240 seconds]

14:35 antocuni has quit [Ping timeout: 256 seconds]

14:38 Taggnostr2 has joined #pypy

14:39 Taggnostr3 has quit [Ping timeout: 248 seconds]

14:43 Taggnostr3 has joined #pypy

14:44 Taggnostr2 has quit [Ping timeout: 255 seconds]

14:47 Taggnostr2 has joined #pypy

14:48 Taggnostr3 has quit [Ping timeout: 256 seconds]

14:59 adamholmberg has joined #pypy

15:08 Taggnostr3 has joined #pypy

15:09 Taggnostr2 has quit [Ping timeout: 264 seconds]

15:17 Taggnostr2 has joined #pypy

15:18 Taggnostr3 has quit [Ping timeout: 260 seconds]

15:36 ceridwen has quit [Ping timeout: 260 seconds]

15:36 lritter has joined #pypy

15:55 ceridwen has joined #pypy

16:27 hotpot33 has joined #pypy

16:31 antocuni has joined #pypy

16:54 adamholmberg has quit [Remote host closed the connection]

16:54 adamholmberg has joined #pypy

16:54 adamholmberg has quit [Remote host closed the connection]

17:17 lritter has quit [Remote host closed the connection]

17:25 antocuni has quit [Ping timeout: 240 seconds]

17:40 hotpot33 has quit [Remote host closed the connection]

17:55 adamholmberg has joined #pypy

18:00 zmt01 has joined #pypy

18:02 kbtr has quit [Ping timeout: 260 seconds]

18:02 dstufft_ has joined #pypy

18:04 kcr has quit [Ping timeout: 245 seconds]

18:04 irclogs_io_bot has quit [Ping timeout: 245 seconds]

18:04 kcr has joined #pypy

18:04 zmt00 has quit [Ping timeout: 245 seconds]

18:04 kbtr_ has joined #pypy

18:04 dstufft has quit [Ping timeout: 245 seconds]

18:04 dstufft_ is now known as dstufft

18:04 kcr has joined #pypy

18:04 kcr has quit [Changing host]

18:05 irclogs_io_bot has joined #pypy

18:06 agates has quit [Ping timeout: 245 seconds]

18:06 dash has quit [Ping timeout: 256 seconds]

18:07 bendlas has quit [Ping timeout: 260 seconds]

18:07 yuvipanda has quit [Ping timeout: 256 seconds]

18:07 danieljabailey has quit [Ping timeout: 240 seconds]

18:08 sthalik has quit [Ping timeout: 240 seconds]

18:09 danieljabailey has joined #pypy

18:09 sthalik has joined #pypy

18:12 Hasimir has quit [Read error: Connection reset by peer]

18:17 Hasimir has joined #pypy

18:29 zmt01 has quit [Quit: Leaving]

18:29 zmt00 has joined #pypy

18:52 jamesaxl has joined #pypy

19:08 exarkun has quit [Quit: ZNC 1.6.1 - http://znc.in]

19:50 exarkun has joined #pypy

19:51 <bbot2> Exception: http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/3958 [py3.5]

19:53 <tsutsumi> Hey there, I'm trying out running pypy3 in production, and I've noticed that both memory and CPU consumption have increased with the change

19:54 <tsutsumi> to give some background, my application basically polls messages from zmq (via pyzmq), parses a json object, matches against a list of regexes, then renders a string and sends that to a tcp socket

19:55 <tsutsumi> the increase in memory isn't really surprising to me because, as I understand it, the GC isn't reference counted so collection doesn't happen as aggressively for short-lives objects, and there's probably a JIT component there.

19:55 <tsutsumi> the cpu is a little surprising. What I see is CPU usage increase in a sawtooth pattern, until hitting 100% cpu utilization on the box

19:56 <tsutsumi> to note, I found an issue where we're not really allocating enough memory for pypy: it starts to consume swap pretty heavily after an hour or two

19:56 <tsutsumi> what's a good way to understand this better? and is this expected?

19:57 <simpson> Once you start swapping, you're going to have a bad time.

19:59 <simpson> I don't know enough about PyPy3 to suggest where to start looking. I'd probably look for some sort of heap analysis; I don't know if meliae works on PyPy3 though.

20:01 <tsutsumi> actually, I took another look, and it looks like the cpu increase maxes out at 100% before we start eating swap.

20:02 <tsutsumi> this further confuses me then. In python2.7 we have solid cpu utilization across the board

20:07 hotpot33 has joined #pypy

20:08 <simpson> That is a little interesting.

20:10 <tsutsumi> I'm thinking at this point the benefit of pypy is reduced because it does a lot of work on the c and kernel layer already, pulling from tcp sockets and zmq being written in c

20:10 <tsutsumi> but I'm wondering if the CPU increase has something to do with GC: we create a lot of garbage quickly (since we're reading data, processing it, and not using it afterward)

20:11 <simpson> You can play with the GC configuration: http://doc.pypy.org/en/latest/gc_info.html

20:19 <tsutsumi> thanks! I've seen that, and we've tweaked a couple things. I was just hoping someone's encountered this before and could help guide on what a good next step might be, before we start tweaking a bunch

20:21 tbodt has joined #pypy

20:22 <bbot2> Failure: http://buildbot.pypy.org/builders/pypy-c-jit-linux-armhf-raspbian/builds/1770 [py3.5]

20:35 dash has joined #pypy

20:41 energizer is now known as energizer_

20:41 energizer_ is now known as energizer

21:16 yuvipanda has joined #pypy

21:16 agates has joined #pypy

21:16 bendlas has joined #pypy

21:39 adamholmberg has quit [Remote host closed the connection]

21:39 adamholmberg has joined #pypy

21:39 adamholmberg has quit [Remote host closed the connection]

22:00 <bbot2> Started: http://buildbot.pypy.org/builders/own-linux-s390x/builds/949 [default]