<maasha>
For some reason rbx is much slower - but reading these graphs is sort of confusing: what steps in the graphs indicate bottle necks?
<yorickpeterse>
brixen: ping
<yorickpeterse>
brixen: wait, in case internet shits itself at some point: lets meet outside next break, lets discuss evil plans
<yorickpeterse>
now lets hope I can actually push my stuff
<yorickpeterse>
yisss, pushed
meh` has quit [Quit: I don't want to live on this planet anymore.]
noop has quit [Ping timeout: 255 seconds]
Ori_P has joined #rubinius
noop has joined #rubinius
jnh has joined #rubinius
jnh has quit [Ping timeout: 264 seconds]
carlosgaldino has quit [Ping timeout: 276 seconds]
carlosgaldino has joined #rubinius
<heftig>
maasha: looks IO-bound to me
<cremes>
maasha: i was gonna say the same. in MRI it is IO bound. in rbx, the hot spot is the String#tr! path.
jnh has joined #rubinius
jnh has quit [Ping timeout: 252 seconds]
<maasha>
eh, String#tr! is extremely low level and evil fast?
<maasha>
So the deal here is to make a Unix pipe lookalike in Ruby, so you do command1 | command2 | command3 | ... command1 is a parser and is probably the bottleneck limited by IO - but how to tell that form the prof graphs?
benlovell has quit [Ping timeout: 276 seconds]
<maasha>
I three different implmentations: forks, threads and enumerables. I am timing some real data and it looks like in all cases rbx is 2-3 times slower than MRI - even for the threaded version.
<maasha>
I have *
<cremes>
maasha: let’s look at the rbx prof data first. check out line 129 which says that String#tr_trans is taking 82.7% of CPU.
<cremes>
further, we see in the next columns over that tr_trans takes 15.47 seconds of time for itself and 4018.58 seconds for its children.
<cremes>
that’s out of a 4878 second total run time.
<cremes>
so, in this case, rbx doesn’t have an “evil fast” low level String#tr.
Ori_P has quit [Quit: Computer has gone to sleep.]
<cremes>
looking at lines 142-144 we see it broken down into even more detail. something inside String#tr_trans is quite slow. looks like there are two block forms… one is okay and the other is a slow beast.
<cremes>
there might be an easy “win” here to do some String#tr_trans optimization.
<cremes>
this is great for rbx because now we have some “real world” code (your code!) that shows a perf problem in rbx. we can fix it and everyone’s
<cremes>
code that uses String#tr! gets faster.
<cremes>
hmmm, breaking it down further, look at lines 152-153. looks like Hash#[] might not be fast in this case. lines 162-163 break it down even more.
<cremes>
maybe it’s using a particularly slow path in Hash#[] that we don’t hit much. hard to say without looking at the code. but now we know where to concentrate our efforts.
<cremes>
and since this is ruby (instead of C or C++) then probably anyone can try to fix it.
<cremes>
maasha: i know it’s probably disappointing that your very first attempt at using rbx showed that it’s slower. don’t lose heart!
<cremes>
maasha: this can be fixed!
<maasha>
cremes: Is tr! implemented in rbx as a Ruby enumerator?
<cremes>
maasha: i don’t know. download the source and take a look. i gotta run off to work… i’ll try and check in later.
<maasha>
cremes: so for checking - what is fastes? Hash[obj] or Hash.has_key? obj (this seems confusing also in MRI)
<maasha>
cremes: Right, appreciated it.
<cremes>
maasha: whenever you wonder which one is fastest, write a benchmark and compare them. look at benchmark_suite for rubinius… it’s better than the stdlib ‘benchmark’
<maasha>
cremes: would be nice if you could simultaneously benchmark across MRI and rbx
<maasha>
Actually, I did benchmark hash[obj] vs hash.has_key? obj in MRI and the first won (Ruby 1.9.x) - Maybe I should try again.
<cremes>
maasha: you can. take a look at benchmark_suite… you can bench across mri, rbx and jruby and compare them. really gotta run… hopefully someone else can give you some guidance this morning!
<maasha>
Great. Thanks
<maasha>
hash[obj] is slightly faster than hash.has_key? obj in MRI. Using rbx hash.has_key? obj is makedly faster thatn hash[obj].
<maasha>
But clearly a premature optimerization thing
<maasha>
yup, tr_trans is looking very slow
<maasha>
bejeeesus, no wonder why it is running like a raped ape :o( - tr is supposed to be in C and the lookup table compiled before runtime.
<maasha>
yopp: OK, so I didn't do a realistic benchmark, but thing is that Hash is slowish compared to MRI.
<headius>
I never thought I'd see the day that tr performance mattered to someone
<headius>
I think I implemented that in jruby
<yopp>
hash[] 5386950.8 (±7.6%) i/s - 26762148 in 4.999343s
<yopp>
hash[] 7221757.4 (±15.4%) i/s - 35279608 in 5.013160s
<yopp>
25%
<maasha>
headius: so in bioinformatics we deal a lot with strings of DNA which quality scores encoded in ASCII. tr is very handy for switching between the different quality score encoding types.
<headius>
yopp: 25% what?
<headius>
maasha: I think that's the first time someone's given me a good example of using tr, too :-)
<headius>
I know some others using Ruby for bioinformatics
<maasha>
headius: yeah, converting DNA to RNA is tr/Tt/Uu/ as an example
kfpratt has joined #rubinius
<maasha>
headius: also reverse-complementing DNA is done with tr.
<maasha>
headius: and as mentioned - converting quality score encodings, but also truncating out of bounds scores.
<maasha>
I luve tr because it is so damned fast - until now!
<yopp>
headius, slower
<headius>
yopp: rbx is slower or MRI os slower?
<yopp>
rbx slower ;)
<headius>
maasha: that's pretty cool...do you have any isolated tr use I could try running?
<maasha>
headius: so a chunk of real life data and a small test script?
<headius>
yopp: ahh
<headius>
maasha: yeah sure
<maasha>
I can cook that up.
jnh has joined #rubinius
<headius>
cool
<headius>
I can't use JRuby if it has inline C of course :-) mostly interested in how you're using tr
<maasha>
headius: this particular test data I have no idea what the project is about. But I have plenty of other interesting stuff: locate bacteria that gives asthma, novel bacteria form the Mariana Trench etc.
<headius>
very nice
<headius>
sigh...I need to port narray to jruby one of these days
<headius>
that will prevent me from running this on JRuby in any case, and I suppose you have MRI numbers already
Ori_P has quit [Quit: Computer has gone to sleep.]
kfpratt has quit [Remote host closed the connection]
kfpratt has joined #rubinius
kfpratt has quit [Ping timeout: 246 seconds]
rafacv has joined #rubinius
<maasha>
headius: yes, biopieces uses inline_C. I have no idea about java, so inline_java is not going to happen.
<headius>
yeah, I didn't think so :-)
<maasha>
inline_c is very handy.
<maasha>
no need to worry about compilation and you have the ruby api at your hands.
<maasha>
and for certain speed critical things you get your 1000 fold speedup
<headius>
yes, it's unfortunate that the Ruby API is so invasive...makes it very hard to support on modern VMs
<maasha>
invasive?
<maasha>
you mean stay away form the API because your code will break stuff else-where?
noop has quit [Quit: Leaving]
<headius>
I mean it depends on being able to access pointers directly...VMs like rbx and jvm can't do that because they move objects for more efficient GC
<headius>
rbx supports C exts by giving you a pointer to a separately-managed handle that can always find the object on the heap
|jemc| has joined #rubinius
kfpratt has joined #rubinius
<maasha>
oh, I think I will just stick with my DNA
<headius>
probably easier than implementing a VM :-)
flavio has quit [Quit: WeeChat 0.4.1]
<yopp>
btw, maasha, can you recommend any good literature, like dunno "Build you own monster using DNA in 21 days" from PragProg?
maasha has quit [Ping timeout: 246 seconds]
meh` has joined #rubinius
jnh has joined #rubinius
jnh has quit [Ping timeout: 260 seconds]
kfpratt has quit [Remote host closed the connection]
kfpratt has joined #rubinius
kfpratt has quit [Ping timeout: 260 seconds]
amsi has joined #rubinius
<|jemc|>
is there any reason why a Module#dynamic_method would not be JIT'd?
<headius>
you're using dynamic_method?
<|jemc|>
it doesn't seem to me like it would make a difference, but - I may be missing something
<|jemc|>
headius: yes, well, I'm playing around with using it anyway
<|jemc|>
it may or may nto remain
<|jemc|>
*not
<headius>
I'm pretty sure evan did no optimization of dynamic_method
<headius>
not that you're concerned about that
<|jemc|>
can you elaborate on what you mean by 'no optimization'?
<headius>
oh, maybe I'm thinking of something else
<headius>
he had a dynamic call site thingy where you could define your own call site logic
<|jemc|>
(and you're right that performance isn't my number one priority but the more little good decisions I can make along the way means less optimization I'm having to do later)
<headius>
yeah, ignore me, I found it
<|jemc|>
dynamic_method is just a convenient way to define a method in terms of bytecode instead of ruby
<headius>
this is just a way to define a method on the fly using the rbx generator
<headius>
yeah
<|jemc|>
yeah
<headius>
as for jitting...dunno why it wouldn't
<headius>
maybe it's not hot enough
<|jemc|>
(I don't have any data to suggest that it is or isn't JIT-ing, just trying to make sure I wasn't missing some obvious reason why it wouldn't before I commit this up)
<headius>
that's not something you'd be hitting on a hot path, right?
<|jemc|>
oh, it's going to be hot :P
<|jemc|>
it's part every method call in my language
<|jemc|>
err... *part of
diegoviola has quit [Quit: WeeChat 1.0]
benlovell has quit [Ping timeout: 246 seconds]
<headius>
you define a dynamic_method for every call?
<headius>
that doesn't seem efficient :-)
<headius>
or are you referring to methods generated via dynamic_method?
<|jemc|>
oh, the definition itself is not part of every call
<|jemc|>
but each call passes through a method generated via dynamic method, yes
<|jemc|>
that does some extra things like deciding whether to actually invoke the real method body or use a cached value, for instance
<headius>
yeah I'd be surprised if they wouldn't JIT
<headius>
hmmm...not that JRuby has its own bytecode-like representation, maybe we should add something like this
<maasha>
yopp: someone already took the bird flu virus and made it truly deadly in the name of science.
<maasha>
It was published earlier this year.
kfpratt has joined #rubinius
meh` has quit [Quit: I don't want to live on this planet anymore.]
GitHub27 has joined #rubinius
<GitHub27>
[rubinius] gustavotemple opened pull request #3131: PowerPC64 support for LLVM and Atomic (master...support-llvm-ppc64) http://git.io/t9Sbng
GitHub27 has left #rubinius [#rubinius]
<headius>
neat
<headius>
does the JIT work on ppc?
<headius>
I thought I heard rbx would need to move to MCJIT for non-x86
elia has joined #rubinius
jnh has joined #rubinius
jnh has quit [Ping timeout: 245 seconds]
|jemc| has joined #rubinius
<|jemc|>
headius: that's true for arm, dunno about ppc
<headius>
yeah I knew ARM for sure
<headius>
oh, looks like ppc and ppc64 are in the old jit too, that's good
<headius>
hmm, third hit in my search says it's not really maintained
<headius>
but it's an oldish post
amsi has quit [Ping timeout: 264 seconds]
<headius>
*shrugs*
|jemc| has quit [Read error: Connection reset by peer]
|jemc| has joined #rubinius
maasha has quit [Quit: Page closed]
Ori_P has joined #rubinius
meh` has joined #rubinius
gtemple has left #rubinius [#rubinius]
jnh has joined #rubinius
jc00ke has quit [Ping timeout: 276 seconds]
jnh has quit [Ping timeout: 272 seconds]
rafacv has quit [Read error: Connection reset by peer]
rafacv has joined #rubinius
jc00ke has joined #rubinius
amsi has joined #rubinius
jc00ke has quit [Ping timeout: 255 seconds]
havenwood has quit []
amsi has quit [Ping timeout: 240 seconds]
|jemc| has quit [Ping timeout: 276 seconds]
|jemc| has joined #rubinius
jc00ke has joined #rubinius
jc00ke has quit [Client Quit]
lopex has quit [Ping timeout: 268 seconds]
dlackty__ has quit [Ping timeout: 268 seconds]
Ori_P_ has joined #rubinius
dmilith has quit [Ping timeout: 268 seconds]
Guest43929 has joined #rubinius
Ori_P has quit [Read error: Connection reset by peer]
dmilith has joined #rubinius
dlackty__ has joined #rubinius
jeregrine has quit [Ping timeout: 268 seconds]
machty has quit [Ping timeout: 268 seconds]
amsi has joined #rubinius
elia has quit [Ping timeout: 276 seconds]
tarcieri has quit [Ping timeout: 276 seconds]
tarcieri_ has joined #rubinius
machty has joined #rubinius
jeregrine has joined #rubinius
elia has joined #rubinius
Guest43929 is now known as lopex
jnh has joined #rubinius
jnh has quit [Ping timeout: 260 seconds]
blowmage` has joined #rubinius
guilleiguaran__ has quit [Ping timeout: 240 seconds]
guilleiguaran___ has joined #rubinius
mrb_bk has quit [Ping timeout: 240 seconds]
Bish_ has joined #rubinius
djinni has quit [Ping timeout: 240 seconds]
eregon_ has joined #rubinius
mrb_bk has joined #rubinius
ssedov has joined #rubinius
fbernier_ has joined #rubinius
Ori_P_ has quit [Quit: Computer has gone to sleep.]
djinni has joined #rubinius
blowmage has quit [*.net *.split]
mpapis has quit [*.net *.split]
alexsuraci has quit [*.net *.split]
fbernier has quit [*.net *.split]
Rotonen has quit [*.net *.split]
jeremyevans has quit [*.net *.split]
chrisseaton has quit [*.net *.split]
stass has quit [Read error: Connection reset by peer]
|Blaze|_ has joined #rubinius
fbernier_ is now known as fbernier
mpapis has joined #rubinius
jfredett-w1 has joined #rubinius
alexsuraci has joined #rubinius
flori_ has quit [Ping timeout: 260 seconds]
flori has joined #rubinius
carlosga_ has joined #rubinius
wca_ has joined #rubinius
jfredett-w1 has quit [Ping timeout: 255 seconds]
febuiles_ has joined #rubinius
eregon__ has joined #rubinius
|jemc|_ has joined #rubinius
blowmage` has quit [Ping timeout: 255 seconds]
|jemc| has quit [Ping timeout: 255 seconds]
kfpratt has quit [Ping timeout: 255 seconds]
wca has quit [Ping timeout: 255 seconds]
|jemc-bot| has quit [Ping timeout: 255 seconds]
eregon_ has quit [Ping timeout: 255 seconds]
chrisseaton has joined #rubinius
carlosgaldino has quit [*.net *.split]
febuiles has quit [*.net *.split]
|jemc|_ is now known as |jemc|
rafacv has quit [*.net *.split]
cremes has quit [*.net *.split]
eregon has quit [*.net *.split]
Bish has quit [*.net *.split]
|Blaze| has quit [*.net *.split]
atambo has quit [*.net *.split]
RealMarc has quit [*.net *.split]
jfredett-w has quit [*.net *.split]
blowmage has joined #rubinius
kfpratt has joined #rubinius
dmilith_ has joined #rubinius
wca_ has quit [*.net *.split]
mrb_bk has quit [*.net *.split]
dlackty__ has quit [*.net *.split]
dmilith has quit [*.net *.split]
lopex has quit [*.net *.split]
johnmuhl has quit [*.net *.split]
halorgium has quit [*.net *.split]
Guest85414______ has quit [*.net *.split]
wca has joined #rubinius
|jemc-bot| has joined #rubinius
johnmuhl has joined #rubinius
febuiles_ has quit [Quit: febuiles_]
kagaro has quit [Ping timeout: 246 seconds]
|jemc| has quit [Ping timeout: 246 seconds]
tarcieri_ has quit [Ping timeout: 246 seconds]
_ko1 has quit [Ping timeout: 246 seconds]
atambo has joined #rubinius
_kfpratt has joined #rubinius
eregon__ has quit [Ping timeout: 246 seconds]
elia has quit [Ping timeout: 246 seconds]
halorgium has joined #rubinius
Guest85414______ has joined #rubinius
kfpratt has quit [Ping timeout: 246 seconds]
rafacv has joined #rubinius
cremes has joined #rubinius
dlackty__ has joined #rubinius
mrb_bk has joined #rubinius
carlosga_ has quit [Ping timeout: 276 seconds]
Caius has quit [Ping timeout: 276 seconds]
Bish_ has quit [Ping timeout: 276 seconds]
Caius has joined #rubinius
Caius has quit [*.net *.split]
carlosgaldino has joined #rubinius
Caius has joined #rubinius
eregon has joined #rubinius
tarcieri has joined #rubinius
Caius has quit [Changing host]
Caius has joined #rubinius
atambo has quit [Ping timeout: 252 seconds]
_kfpratt has quit [Remote host closed the connection]
dmilith_ has quit [Ping timeout: 252 seconds]
|Blaze|_ has quit [Ping timeout: 252 seconds]
atambo has joined #rubinius
amsi has quit [Ping timeout: 245 seconds]
amsi has joined #rubinius
CaptainJet has joined #rubinius
tenderlove has joined #rubinius
<postmodern>
thinking about splitting the md5s/version files out of ruby-install and into a separate git repo
<postmodern>
would anyone be interested in maintaing the rbx files?