<enebo>
kares: 9.1 should always be ok to merge or did headius land something which gave you pause?
jrafanie has joined #jruby
xardion has quit [Remote host closed the connection]
xardion has joined #jruby
LiamW has joined #jruby
<kares>
enebo: hey - he was into that DynaMethod.name refactoring, he noted its still not finished
<enebo>
kares: ah ok yeah I know we talked about notion of adding a version anno to invokers
<kares>
mostly last time we chatted he wasn't sure whether its not going to get reverted
<LiamW>
installed graalvm+truffleruby for the hell of it, and due to some unrelated factors i got a kernel panic while installing a gem - now ruby won't start
<enebo>
kares: so we can check which constructor to call
<enebo>
kares: then old code will still work but have weird anonymous names but newer invokers will have proper name constructor
<kares>
aah oki
<kares>
so what do you say should should we wait for the merge?
<enebo>
kares: so I guess that is probably still happening but headius got sick while travelling so it might delay that
<kares>
aah bummer :(
<kares>
no biggie - my concern is to have CI green
<enebo>
kares: I would merge. Worst case as he updates he has to change something in a later merge
<enebo>
kares: I would rather be green
<kares>
and I fixed some of the .name stuff on 9.1 - hoping to get it over
<kares>
okey I will do - likely tomorrow - unless you beat me to it today ;)
<enebo>
kares: yeah we should prioritize CI so it is simpler for people to contribute
<enebo>
kares: yeah I am still mired on bytelist work. I now hit a new wrinkle in idutils needing to work with mbc identifiers
<kares>
definitely + I am also checking it while doing work and its annoying to have to check manually for the red builds
<enebo>
I think once I make it past that point my branch will be green and largely work for all mbc encoding cases
<kares>
actually have checked your branch today - just so I know some parts :)
<kares>
wasn't sure whether Symbol will keep their String repr. around or only ByteList
<enebo>
kares: the raw string then perhaps will need to add interning but RubySymbol has it as a hard field
<enebo>
err field
<enebo>
so I think it should stay so long as the symbol itself does
<enebo>
I could possibly see something I missed in that but that could just mean hard symbol path again
<enebo>
in MRI all identifier IDs end up being eagerly made and not GCd ever
<enebo>
or the symbols associated with them anyways
<kares>
yep I know that
<kares>
JRuby pretty much had a better setup with Symbols since inception
<enebo>
so ID is raw 8859_1 String in bytelisT_love
<enebo>
with value in symbol table being properly encoded bytelist
<enebo>
well symbol but whatevs
<enebo>
So it will be more like MRI now though :)
<kares>
yep - that what I ended up pretty much following on the branch
<lopex>
enebo: so it's still susceptible to starvation ?
<enebo>
lopex: you mean if you endlessly create new identifiers?
<lopex>
yeap
<enebo>
lopex: yeah MRI is too
shellac has quit [Quit: Leaving]
<lopex>
enebo: so any json which gets keys symbolized ?
<lopex>
being fed from outside
<enebo>
lopex: JSON are not identifiers per se
<kares>
those symbols aren't stored
<enebo>
lopex: even if it uses some method_missing sort of thing those symbols would still collect
<enebo>
lopex: the problem my work may have is I might not have actually made non-weak entries for symbols
<LiamW>
okay nvm
<LiamW>
found it
<LiamW>
proc wasn't mounted
<enebo>
lopex: I did think I could make all symbols weak if I stored symbol in IR instrs but then I have always wanted to evacuate IR code to a persistent store
<LiamW>
it was trying to read /proc/self
<enebo>
lopex: If I did that then those symbols would disappear
<enebo>
chrisseaton: with graal and our math identity patch we probably pass them back up on this one
<LiamW>
>TruffleRuby is over 300x faster
<LiamW>
lol
<chrisseaton>
Yeah, it's all about the boxing here - if you have any you're not going to do well
<enebo>
but that obviously is not something we can run with right now
<chrisseaton>
I thought IR could eliminate boxing?
<enebo>
chrisseaton: yeah the patch removes needing identity for Fixnum so Graal reduces it all to long math
<enebo>
chrisseaton: well we have code which has done that but we are not using it atm
<enebo>
chrisseaton: Not so much that we have given up but it seems real life support combined with our resource load has pushed that back
<enebo>
chrisseaton: when we enabled those instrs we did speed up mandlebrot by like 5-7x or something like that
<chrisseaton>
I was going to ask about that patch - it seems to remove identity for all classes in the JRuby package - is that right?
rrutkowski has quit [Ping timeout: 268 seconds]
<enebo>
chrisseaton: but I think we were missing guards
<enebo>
chrisseaton: so there's that
<enebo>
chrisseaton: no only float and fixnum types
<enebo>
chrisseaton: since those have no reason to need object identity
<chrisseaton>
Rubinius isn't doing well here - Truffle is 3000x faster than Rubinius
<enebo>
It is possible headius added a few others
<enebo>
but I expect MJIT to do well on math
<enebo>
I don't have a lot of expectations beyond that
<enebo>
but that is worth it alone
<enebo>
oh lunch...bbiab
<chrisseaton>
Yes, as I've said in a few talks - it's all about inlining deep through the core library, out the other end, back into user code, and back into the core
<chrisseaton>
Unless you do that... it doesn't matter how fast you make anything because it'll all just be runtime calls
<LiamW>
mhm
<LiamW>
ruby is method calls all the way down
<chrisseaton>
LiamW: my canonical example is [a, b, c].sort[1] - you need to compile that so it makes no calls and allocates no objects - that's how to make idiomatic Ruby fast I think
<LiamW>
there isn't even any sort of parser rule for e.g. basic addition (though MRI specializes opt_plus to optimize it), it has to be implemented in C
<lopex>
btw is there a lot of pressure in compiler development for cache utilization ?
<chrisseaton>
Well you can also implement it inline in assembly code in the compiled method - that's what TruffleRuby does
<LiamW>
sorry, I'm sort of speaking in the context of MRI here
<chrisseaton>
Cache is more important than it used to be yes, but cache oblivious is old
<LiamW>
you could in theory implement Fixnum#+ as something like def +(other); __builtin_fixnum_add(self, other); end (where the __builtin_* is a parser rule that outputs an addition instruction)
<LiamW>
but ruby is so thoroughly magic that you're going to need a lot of those rules
<lopex>
mri has a global table for those depots
<lopex>
I guess refinements complicate it more
<LiamW>
my point is that mri ruby doesn't implement them as parser rules
<lopex>
doh, "ß".upcase kills so many iso_* cr optimizations
<cremes>
trying to track down another bug in jruby’s SortedSet implementation…
<cremes>
is this stored in the jruby tree or is it loaded as a gem from elsewhere?
<cremes>
anyone know?
<lopex>
looks like TreeSet
<cremes>
lopex: I meant to ask, “is the ‘set’ gem stored in the jruby source tree?"
<lopex>
cremes: under ext
<cremes>
I know underneath it’s using ‘rbtree’
<cremes>
ok will look… thanks
<cremes>
oh crap, this means it is implemented as java
<cremes>
I have a scenario where adding an object to a sorted set using #add? returns self (which means the add succeeded and was not a dupe), increments the size of the set, but then does NOT actually add the object. Using set.to_a.size shows that the set didn’t grow.
<cremes>
I just need a repro… i was hoping the source would give me a hint as to how it’s comparing identities
<cremes>
looking...
<kares>
cremes: what JRuby version?
<kares>
master did a SortedSet rewrite in native
<cremes>
master as of Jan 31
<kares>
otherwise its set.rb under stdlib
<kares>
okay than its RubySortedSet.java
<kares>
set.to_a had a bug lately
<kares>
ah you were the one to report :)
<cremes>
yep!
<cremes>
found another one but making a repro looks hard
<cremes>
hard as in “I don’t know what details to capture to prove the problem"
<kares>
;(
<cremes>
I caught this issue when verifying the set was growing after an add
<cremes>
it said “yes” but converting to an array said "no"
<kares>
well any kind of .rb script is helpful - if you can share in a bug report
<cremes>
I see from the impl that #add adds to both a hash and a tree. when running #to_a, it utilizes ONLY the tree to output the array
<kares>
no need to minimize down if its proving difficult
<kares>
yep there's a trick but it is expected to work
<cremes>
sure
<cremes>
the library should add an internal assert… assert that hash.size() == order.size()
<cremes>
after every dadd
<cremes>
I bet that would catch it quickly
<kares>
yep we can do that
<cremes>
I’ll do that in my local copy… what should I output when it fails to help track down WHY?
<kares>
I am not sure we want a lot of details printed
<kares>
usually a plain old Java assert is fine
<kares>
really a reproducer is best - its easy to fix it from there when there's a deterministic .rb script
<cremes>
kares: like this? ‘assert size() == order.size();'
subbu is now known as subbu|afk
<LiamW>
chrisseaton: when you deoptimize in truffleruby, how do you know exactly what state to return to the interpreter if everything should have been optimized away? (e.g., what registers match up to what variables, what the current call stack is, etc.)
<LiamW>
is there a separate kind of "debug info" that gets stored with compiled code to match it up?
<chrisseaton>
Yes it’s like debug info
<cremes>
kares: I added this and it won’t compile: assert size() == to_a.size();
<cremes>
kares: help!
subbu|afk is now known as subbu
claudiuinberlin has joined #jruby
<cremes>
this compiles: assert size() == to_a(runtime.getCurrentContext()).size();
<cremes>
but it doesn’t actually raise when things get out of sync… is there something else special to do?
<GitHub49>
[jruby] chuckremes opened issue #5035: JRuby's native `SortedSet` lies about adding an object https://git.io/vAJ24
<LiamW>
I meant more that my opencv usage is limited to something I could easily recreate with graphicsmagick
<LiamW>
just haven't yet
<LiamW>
chrisseaton: installed successfully
<chrisseaton>
Bingo
<chrisseaton>
So it's either the fact that it's C++, or something else in it
<lopex>
chrisseaton: btw does sulong suffer any cold startup timing issues ?
<chrisseaton>
As much as TruffleRuby does - but it's pretty much a solved problem now
<chrisseaton>
We start as fast as MRI does - faster than JRuby and Rubinius
<lopex>
oh cool
<chrisseaton>
So we can tick another 'well but' off the list!
<lopex>
well, you need more PR now for it
<lopex>
dont have to convince anyone whos interested
<LiamW>
when you are building native extensions with truffleruby, are they being compiled to llvm ir on disk? if that's true, is that then being read and executed by sulong when you go to use the extension?
<LiamW>
(forgive me, I'm just figuring out how this all fits together)
<chrisseaton>
Yes on disk - they live temporarily in `.bc` files, but they're linked into `.su` files.
<chrisseaton>
A `.su` is just a `.jar` of `.bc` really.
<chrisseaton>
Yes Sulong reads the bitcode and executes it much like TruffleRuby reads and executes Ruby code.
<lopex>
LiamW: the point being having it all in the same ast so one thing optimizes inlines etc
<LiamW>
well, yes, I got that from the blogposts
<LiamW>
I got the same error as before installing hitimes
<LiamW>
hitimes doesn't use C++, only C
<chrisseaton>
That's a pretty simple one - I'll look at it some time
drbobbeaty has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<nirvdrum>
I'm hopping in mid conversation here, so I'm probably missing context. We do support C++ extensions in some capacity. E.g., "unf_ext" builds and runs fine.
<LiamW>
removing ruby.bc and trufflemock.bc from the command line allows it to compile
<LiamW>
which suggests they need to be regenerated
<LiamW>
how do I do that
<LiamW>
chrisseaton: ^
<nirvdrum>
I'll bring up a jessie Docker image and see if I can reproduce.
<nirvdrum>
LiamW: Although it looks like you're right. Sulong can handle LLVM 3.8 and 4.0 just fine. But it looks like these .bc files were generated with clang 4.0 and the 3.8 tools can't handle them.
<nirvdrum>
At least that's what it looks like to me at first blush. I've never really tried before so I'll need to investigate more.
<nirvdrum>
But that's obviously a lot more involved than just downloading a tarball.
<chrisseaton>
Hmm we ship the IR don't we
<chrisseaton>
I'll look into this, thanks
<LiamW>
I'm not that immediately motivated because I can't test any of this on my local box
<LiamW>
cpu is toasted from a heatsink failure and causes MCEs through the roof whenever I do anything with a lot of locking
<LiamW>
running jruby causes kernel panics
<LiamW>
therefore I'm running all of this on a server
<LiamW>
nirvdrum: it'd be greatly appreciated if you could build the native stuff against 3.8 and throw me a tarball but if you don't want to I totally get that
<chrisseaton>
He just needs to send you a couple of .su files