04:24 <travis-ci> The build is still failing. by @usa: See https://travis-ci.org/ruby/ruby/builds/269058715

07:54 <znz_jp> biff: [ruby-changes:47555] glass:r59671 (trunk): lib/csv.rb: refactor and optimize. This change - http://mla.n-z.jp/?ruby-changes=47555

08:06 <travis-ci> The build was broken. by @glass: See https://travis-ci.org/ruby/ruby/builds/269095012

08:31 <znz_jp> biff: [ruby-changes:47556] hsbt:r59672 (trunk): Merge rubygems-2.6.13. - http://mla.n-z.jp/?ruby-changes=47556

08:37 <travis-ci> The build was fixed. by @hsbt: See https://travis-ci.org/ruby/ruby/builds/269105114

10:55 <znz_jp> biff: [ruby-changes:47557] glass:r59673 (trunk): string.c: fix potential bug in String#split - http://mla.n-z.jp/?ruby-changes=47557

11:20 <znz_jp> biff: [ruby-changes:47558] kazu:r59674 (trunk): [DOC] Fix typo in rdoc of `transform_values!` [ci skip] - http://mla.n-z.jp/?ruby-changes=47558

14:21 <znz_jp> biff: [ruby-changes:47559] nobu:r59675 (trunk): compile.c: compile_if - http://mla.n-z.jp/?ruby-changes=47559

16:05 <znz_jp> biff: [ruby-changes:47560] naruse:r59676 (trunk): Increase STACKFRAME_FOR_CALL_CFUNC - http://mla.n-z.jp/?ruby-changes=47560

16:05 <znz_jp> biff: [ruby-changes:47561] svn:r59677 (trunk): * 2017-08-29 - http://mla.n-z.jp/?ruby-changes=47561

19:03 <tenderlo_> jashmatthews: hello!

19:04 <jashmatthews> Hey! Wow, I have not used IRC in about 10 years. I feel like such a noob

19:04 <tenderlove> hahaha

19:04 <tenderlove> we can switch to Slack, I just don't know how to invite people

19:04 <jashmatthews> all good

19:04 <tenderlove> so you're just measuring with GC.stat and /proc?

19:04 <jashmatthews> yeah

19:05 <tenderlove> hmmmm

19:05 <tenderlove> not sure if I can access /proc on our production hosts

19:05 <tenderlove> so the way I've been measuring is by logging all malloc call, and their stack traces

19:05 <tenderlove> for a basic Rails app, I was only seeing like 18% (or so) of the mallocs go to heap pages

19:06 <tenderlove> For GitHub, it was like 30%, but that's because we increase the heap size at boot

19:06 <jashmatthews> does it matter that direct malloc is malloced and freed

19:06 <tenderlove> the book keeping takes care of that

19:06 <tenderlove> it logs all malloc / free

19:06 <tenderlove> hmmmm

19:07 <tenderlove> what are you looking at in /proc

19:07 <tenderlove> smaps?

19:07 <jashmatthews> yeah

19:07 <jashmatthews> just to measure SharedClean and Dirty

19:08 <jashmatthews> I'm using RSS for the total memory to work out Ruby heap vs total heap

19:08 <tenderlove> huh

19:08 <tenderlove> what kind of increases did you see in SharedClean?

19:08 <jashmatthews> almost nothing! but SharedDirty stopped dropping by ~85MB

19:10 <tenderlove> huh

19:13 <tenderlove> trying to get GC stats from our prod boxes

19:15 <tenderlove> annoying

19:15 <tenderlove> I can't get to SharedDirty

19:15 <tenderlove> I need to ask ops people

19:15 <tenderlove> ugh

19:17 <tenderlove> jashmatthews: I'll have to get back to you. But I'd like to compare our stats with yours (and measure the same thing)

19:18 <jashmatthews> no problem!

19:19 <jashmatthews> I wonder if anyone has published GC.stats in blog posts

19:19 <jashmatthews> maybe Discourse or someone like that?

19:25 <jashmatthews> https://samsaffron.com/archive/2013/11/22/demystifying-the-ruby-gc

19:25 <jashmatthews> GC Stats: count : 114, heap_used : 4283, heap_length : 4312, heap_increment : 0, heap_live_num : 459148, heap_free_num : 1283203, heap_final_num : 0, total_allocated_object : 6292870, total_freed_object : 5833722

19:45 <jashmatthews> tenderlove: https://github.com/jamatthews/ruby/commit/abca69cfc347f37c5a43dd2535122b0922fb93b7

19:45 <jashmatthews> this was my proof of concept

19:46 <tenderlove> nice!

19:46 <jashmatthews> but I'm not sure it actually works

19:46 <jashmatthews> I think it might still be allocating into existing pages

19:46 <jashmatthews> just allocating later in the heap, into less full pages

19:48 <jashmatthews> it's just a horrible hack really

19:48 <jashmatthews> now I'm trying to figure out how to swap to a fresh heap after forking

19:49 <jashmatthews> but getting all the GC stuff to work properly is a nightmare

19:52 <jashmatthews> if I turn this CoW barrier on then fork

19:52 <jashmatthews> I see this

19:52 <jashmatthews> before trying to fill the slots: {:count=>49, :heap_allocated_pages=>2381, :heap_sorted_length=>4273, :heap_allocatable_pages=>1888, :heap_available_slots=>970492, :heap_live_slots=>690935, :heap_free_slots=>279557, :heap_final_slots=>0, :heap_marked_slots=>690934, :heap_swept_slots=>1049862, :heap_eden_pages=>2381, :heap_tomb_pages=>0, :total_allocated_pages=>2384, :total_freed_pages=>3, :total_allocated_objects=>4444515, :total

19:52 <jashmatthews> _freed_objects=>3753580, :malloc_increase_bytes=>1104, :malloc_increase_bytes_limit=>26868640, :minor_gc_count=>35, :major_gc_count=>14, :remembered_wb_unprotected_objects=>18383, :remembered_wb_unprotected_objects_limit=>36766, :old_objects=>654985, :old_objects_limit=>1309970, :oldmalloc_increase_bytes=>1552, :oldmalloc_increase_bytes_limit=>25743200}

19:52 <jashmatthews> after "filling" the slots but getting blocked by the barrier

19:52 <jashmatthews> {:count=>49, :heap_allocated_pages=>3068, :heap_sorted_length=>4273, :heap_allocatable_pages=>1201, :heap_available_slots=>1250516, :heap_live_slots=>970997, :heap_free_slots=>279519, :heap_final_slots=>0, :heap_marked_slots=>690934, :heap_swept_slots=>1049862, :heap_eden_pages=>3068, :heap_tomb_pages=>0, :total_allocated_pages=>3071, :total_freed_pages=>3, :total_allocated_objects=>4724577, :total_freed_objects=>3753580, :malloc

19:52 <jashmatthews> _increase_bytes=>2305992, :malloc_increase_bytes_limit=>26868640, :minor_gc_count=>35, :major_gc_count=>14, :remembered_wb_unprotected_objects=>18383, :remembered_wb_unprotected_objects_limit=>36766, :old_objects=>654985, :old_objects_limit=>1309970, :oldmalloc_increase_bytes=>2306440, :oldmalloc_increase_bytes_limit=>25743200}

19:53 <jashmatthews> free slots basically stayed the same, which is what I'd expected

19:53 <jashmatthews> but I thought empty pages were in heap_tomb?

19:53 <jashmatthews> what's a heap_allocatable_page vs heap_allocated_page?

19:53 <jashmatthews> is an allocated page full?

19:55 <jashmatthews> eden_pages went up

19:55 <jashmatthews> I don't get it

19:55 <jashmatthews> did I force Ruby to allocate memory without running GC?

23:26 <znz_jp> biff: [ruby-changes:47562] nobu:r59678 (trunk): win32.c: fix function pointer - http://mla.n-z.jp/?ruby-changes=47562