<burnninator>
Yeah, that's the same behavior. So since it's the behavior in cruby, it's either 'as designed' or a bug in cruby. I'll investigate some more.
<headius>
burnninator: ok
<burnninator>
thanks for the help!
<headius>
you can pass true to include non-visible methods, perhaps that's what you want
<headius>
respond_to? :foo, true
<burnninator>
Yeah, that does work. Appreciate the guidance!
<GitHub4>
jruby/master 63be5a4 mh: an example test case for #4770 that shows how unicode readings fails in readonly transaction
<GitHub4>
jruby/master 06aa3f7 Thomas E Enebo: Merge pull request #4771 from duritong/yamlstore-unicode-failure-in-readonly-transaction...
<GitHub123>
[jruby] enebo closed pull request #4771: an example test case for #4770 that shows how unicode readings fails … (master...yamlstore-unicode-failure-in-readonly-transaction) https://git.io/v5B9J
olle has quit [Quit: olle]
<kares>
good reminder - wanted to update the 2.5 tracking issue ... its done now
Puffball has joined #jruby
<GitHub43>
[jruby] kares closed pull request #4422: [#4403] To be consistent with MRI, File.exists? for <path to file> end… (master...4403-fix-exists-method) https://git.io/vMcfd
Puffball has quit [Remote host closed the connection]
dave__ has quit [Remote host closed the connection]
shellac has quit [Quit: Leaving]
olle has joined #jruby
ddadacha has joined #jruby
olle has quit [Quit: olle]
olle has joined #jruby
olle has quit [Client Quit]
<GitHub183>
[jruby] danielford closed pull request #4746: Adding flag to skip calling File.getCanonicalPath() on argv[0] during exec() (master...native-exec-canonicalizesymlinks-option) https://git.io/v5UmP
drbobbeaty has joined #jruby
claudiuinberlin has joined #jruby
drbobbeaty has quit [Read error: Connection reset by peer]
<GitHub35>
jruby/master deee6cc Charles Oliver Nutter: Try bumping memory up for MRI suites to avoid travis hangs.
byteit101_ has quit [Ping timeout: 255 seconds]
ddadacha has quit [Quit: Page closed]
byteit101 has joined #jruby
dave__ has quit []
<GitHub146>
[jruby] headius created memory_bump (+1 new commit): https://git.io/vNOn6
<GitHub146>
jruby/memory_bump 43c5e33 Charles Oliver Nutter: A different memory bump.
<headius>
rtyler: well travis says it takes 4h on whatever mickey mouse instances we get
<headius>
enebo and I are looking into stalls and startingto think these instances are way over-allocated
<enebo>
you see last few tests in output take a lot more time then nothing for 10 minutes
<enebo>
I think it is paging like a mad dog at that point
<headius>
it falls over a cliff
<headius>
seems like memory issue in some way
<enebo>
it is a large memory run too
<rtyler>
how much of a memory?
<headius>
ugh, I'm more and more of the opinion that it's travis
<headius>
last couple lines in a current build:
<headius>
[exec] [1412/8763] TestArray#test_delete2 = 0.00 s
<headius>
[exec] [1413/8763] TestArray#test_delete_at = 0.00 s
<headius>
[exec] [1414/876
<headius>
there's NO way mvn cut out halfway through that write
<headius>
enebo: did you get a peak memory on test:mri?
<enebo>
no I killed it and it was over 1.5GiB though on process monitor (or whatever this is)
<enebo>
man gnome runs like 300 processes
<enebo>
virtual was like 8
<headius>
enebo: I'm well past the array tests and have not exceeded like 250MB heap
<lopex>
so many numbers
<headius>
I don't see a leak
<enebo>
headius: yeah likely I am way past that point
<enebo>
headius: but in any case I am not looking at the heap
<enebo>
headius: I mean not all of java or even jruby is managed
<enebo>
headius: that would be a worse problem if that is an issue obviously
<headius>
ok, I'm looking at the heap right now
<headius>
going up more now
<headius>
500MB
<enebo>
My dumped heap as a file is <500MB but I think it may be compressed
<rtyler>
1.5GB is nothing
<headius>
GC drops it down to 200 and then it zooms back up to 500 occupied
<rtyler>
that's no problem, these agents provision with 8GB by dfefault
<headius>
so craploads of garbage
<headius>
rtyler: nothing we have needs that much, so that's fine
<enebo>
rtyler: but are you saying we get 8GB from travis?
<headius>
ok yeah back down to 200 range now and stable at TestIO
<headius>
so screw you travis
<headius>
and the recent hangs have been happening randomly inside pretty trivial tests
<rtyler>
enebo: no I'm saying the agents Code Valet provisions are much bigger and fatter :)
<enebo>
rtyler: ah ok :)
<headius>
enebo: so yeah, we definitely push into 500MB occupied heap a few times but it always goes back down to 200ish
<headius>
eventually
<headius>
we're pushing heap too big for travis instances that could do it
<rtyler>
are there any other jruby folks heading to fosdem besides you headius?
<headius>
not that I know of
<headius>
kares: FOSDEM?
<headius>
there's no Ruby room this year so that is a bummer
<rtyler>
organizing a devroom sucks, just so you're aware :P
<headius>
I'm sure
<headius>
that's why I've never volunteered to do it
<rtyler>
ruby feels stagnant in europe to me
<headius>
could be
<rtyler>
reminds me a bit of python in that regard, people are still using it, but the language itself isn't significantly innovating, so discussions around the language/vms is isolated to implementors rather than a broader userbase
<rtyler>
I doubt we'd get allocated as a devroom ever again without that dynamic changing
<enebo>
so the new rescuepc code I made is creating lots of garbage
<headius>
enebo: good job
<enebo>
57k int[] many of which are dead
<enebo>
65M
<enebo>
or 14% of whenever I captured this heap
<enebo>
I maybe should reconsider an int[] per ipc
<headius>
I don't see why it would create an unreasonable amount of garbage
<enebo>
evals maybe?
<headius>
testing is a ton of one-off scopes
<headius>
I'd wager you're just seeing that
<enebo>
I don't know why there were so many unreferenced ones
<enebo>
it makes it and stores it in interpretercontext forever
<enebo>
so we have one per scope
<headius>
yeah
<enebo>
I can make this ln(n) search on number of regions and drastically reduce this space
<headius>
hmmm
<headius>
are we calling prepare too much?
<enebo>
then rescue code in startupinterp will do ln(n) array searches
<headius>
maybe it should have a "prepared" flag like some other ones do
<headius>
you cache it but if we're continually calling prepare it will process ahain
<headius>
again
<enebo>
well I could make this lazy for sure but I perform it 100% when instructions are made into an interpretercontext
<enebo>
oh hmm
<headius>
like, are we calling it on every entry or somethjing
<enebo>
ok for that to be true we would need to keep recalling allocateInterpreterContext
<enebo>
and if we were whoo boy
<enebo>
headius: this is a red herring this cannot be rescuePCs
<enebo>
do we use int[] for hashing somewhere?
<enebo>
all the values are all over the place valuewise
<enebo>
ok and this one happens to be backtrace data
<enebo>
weird shit
<enebo>
I keep thinking whoever wrote OQL for visualvm is trolling the world
<enebo>
It takes me like 6 hours to make a proper query
<enebo>
and there is so little documentation on it
<enebo>
we make a Ruby DSL for visualvm and they would probably put JRuby into openjdk
<enebo>
This heap has 225k iso8859_1 decoders in it too
<enebo>
Only reference I have found to it thus far is in a threadlocal from a fiber
<headius>
enebo: we use int[] all over
<headius>
run an allocation profile
<headius>
-Xrunhprof:depth=20
<headius>
very slow
<enebo>
with test:mri :)
<lopex>
enebo: jcodings does hash int[], from, to tripple
<lopex>
but it's local values
<enebo>
lopex: yeah I think I was mostly surprised with how many were there but unreferenced
<enebo>
lopex: I don't frequently examine our short-lived garbage
<enebo>
like in a short period of time we make 225k 8859 charset decoders
<lopex>
that heap was taken just after gc ?
<lopex>
yeah, weird
<enebo>
no just randomly
<enebo>
I was running test:mri and just dumped at some point
<lopex>
does OQL help filtering the roots ?
<enebo>
does OQL help anything
<enebo>
It probably does if you can figure out how to make the query
<enebo>
although that one is a little simpler to me
<lopex>
enebo: is it jcodings decoders ?
<lopex>
enebo: it so they use huge int arrays
<enebo>
lopex: nearly all no longer have any reference
<lopex>
I wonder if the values would tell anything
<enebo>
lopex: ok well there are a fair number which are 32 in length
<headius>
where are those charset decoders from?
<headius>
for UTF8 I cache some decoders per thread to avoid that
<enebo>
as I said I only found one which had a real reference left
<headius>
but not all code calls that
<enebo>
it was from a fiber with a threadlocal to it
<headius>
hmm
<enebo>
I suspect studying this may find a few gems where we alloc something over and over we could share
<enebo>
but then again it is capturing a test suite run
<enebo>
lots of one time stuff
<enebo>
I expect a lot of garbage
<headius>
right
<enebo>
return Charset.forName("ISO-8859-1");
<enebo>
charsetForEncoding
<enebo>
heh so RubyString.toString() basically calls that every time
<headius>
ugh
<lopex>
enebo: hey, btw specialized separate joni interpreter loop for singlebyte with faster ops and now we can hook it to be used when string coderange is 7Bit
<lopex>
enebo: so no bytecode rewrite is needed
<enebo>
lopex: cool
<enebo>
charset decode can be cached can't it?
<enebo>
n threads can use the same one right?
<enebo>
If not the design of java charsets seems like it encourages a lot of garbage
<headius>
well that's an easy fix
<headius>
lopex: nice!
<headius>
ByteList.plain
<headius>
ahh
<headius>
no, decode
<headius>
but we do have a decoder cache
<enebo>
maybe a drop in the bucket compared to making actual string data
<headius>
decoders are definitely not threadsafe
<enebo>
we should make our own :)
<enebo>
8859_1 is about the simplest decoder ever
<enebo>
it could be an arraycopy
<enebo>
headius: but yeah plain() seems more in order