pawnbox has quit [Remote host closed the connection]
yfeldblum has joined #jruby
pawnbox has joined #jruby
<kares>
ahoy!
<GitHub90>
[jruby] juergenthomann opened issue #3980: rbThrowInternal throws error if IRubyObject ID is greater than Integer.MAX_VALUE https://git.io/voHAk
skade has joined #jruby
prasunanand has quit [Ping timeout: 244 seconds]
sandelius has joined #jruby
pawnbox_ has joined #jruby
pawnbox has quit [Ping timeout: 276 seconds]
pawnbox_ has quit [Remote host closed the connection]
pawnbox has joined #jruby
bb010g has quit [Quit: Connection closed for inactivity]
shellac has joined #jruby
yfeldblum has quit [Ping timeout: 250 seconds]
prasun has joined #jruby
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #jruby
<GitHub180>
[jruby] chrisseaton pushed 1 new commit to master: https://git.io/voQTM
<GitHub180>
jruby/master 55392a6 Chris Seaton: [Truffle] Error results for failing benchmarks.
<GitHub73>
[jruby] chrisseaton pushed 1 new commit to truffle-head: https://git.io/voQT9
<GitHub73>
jruby/truffle-head b5f32fd Chris Seaton: Merge branch 'master' into truffle-head
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #jruby
skade has quit [Quit: Computer has gone to sleep.]
skade has quit [Quit: Computer has gone to sleep.]
e_dub has joined #jruby
thedarkone2 has joined #jruby
camlow325 has joined #jruby
prasunanand has joined #jruby
camlow325 has quit [Read error: Connection reset by peer]
camlow325 has joined #jruby
subbu|afk is now known as subbu
<headius>
howdy howdy
<headius>
eregon: SourceSection is a Truffle thing I assume?
<headius>
kinda wondering if there are some utilities like that we'd be able to use for JRuby classic
<chrisseaton>
headius: yeah it's part of our API, it's being redesigned a bit in truffle-head
<chrisseaton>
it doesn't yet give the precision that Ruby needs - byte offsets from the parser rather than character offsets, but we're working on that
<headius>
character offsets for what?
<headius>
is it off on line numbers or something?
<headius>
chrisseaton: I don't suppose Truffle has anything for coroutines, does it? :-)
<chrisseaton>
the Zippy people implemented generators in Python, using a duplicated partial ASTs for each method at every point where you can re-enter
pawnbox has quit [Ping timeout: 246 seconds]
<chrisseaton>
Might work for Ruby enumerators, for which a fibre is spawned at the moment
skade has joined #jruby
<headius>
python generators are extremely limited...even Jython supported them
<headius>
you can't yield down stack
<headius>
that might be a worthwhile case to support but it's a hell of a drop off that cliff when you start yielding in a downstream call and we have to thread it up
pawnbox has joined #jruby
<chrisseaton>
And Ruby enumerators don't make it clear that there is a big cost there
<chrisseaton>
You can easily use them and not realise
<headius>
threaded fibers are still one of the biggest thorns in our side
<headius>
yeah the enumerator thing is insidious
<headius>
we have some specialization so that if it's a simple array it won't spin up a thread, but those one-offs only go so far
<headius>
all it takes is someone mixing in Enumerable and next'ing an Enumerator from it and we have to use a fiber/thread
<headius>
sigh
<eregon>
headius: I have a fork of Truffle with Coruoutines, but you need Graal for native support :p
<headius>
eregon: can it resume arbitrary Java stacks?
<eregon>
(so also a fork of Graal since that's not in JVMCI)
<eregon>
it's just using Thread on non-Graal
<eregon>
I wonder how much this matters for the NES benchmarks which uses Fibers, I should try :)
<headius>
we've come up with ways to implement lightweight coroutines for pure Ruby-to-Ruby call stacks but without JVM support any intervening non-Ruby breaks that
<headius>
and we have to know ahead of time whether to do the fiber under a thread, so it makes the whole idea moot
camlow325 has quit [Read error: Connection reset by peer]
<GitHub99>
[jruby] chrisseaton pushed 1 new commit to truffle-head: https://git.io/voQbk
<GitHub99>
jruby/truffle-head 200eeb0 Chris Seaton: [Truffle] Building Graal 0.12 in error.
<eregon>
How would you do it in the Ruby-to-Ruby case?
camlow325 has joined #jruby
<headius>
trampolined calls
<headius>
at least in the interpreter, and doable in JIT with some work
<headius>
so no ruby-to-ruby calls actually deepen JVM stack once you're under a fiber
<headius>
that's how it would work in theory, anyway...basically we do the work of turning calls in a fiber into continuation-passing
<eregon>
but you have to keep the frames and stacks around then?
<headius>
only artificial stack, but yes
<eregon>
yeah, what I meant
<headius>
no worse than the bytecode-weaving coroutine libs on JVM
<headius>
and no better
<headius>
bottom line for me is that if we can't pause/resume through arbitrary JVM code, it's not even a partial solution...it's an unworkable one :-(
<headius>
the top-level simple fiber thing is the only option that might work, and that requires us to treat Fiber.yield as a keyword
<eregon>
do you often see people using Fibers in perf-sensitive ways??
<chrisseaton>
headius: do you know for a fact that the overhead of the thread solution is really high? it certainly sounds like it would be, but where is the cost? just the context switch each time?
<headius>
it is
<headius>
I've tried everything
<headius>
I've asked concurrency-interest for ideas too and they had nothing
<headius>
it's the scheduler, we believe...you can't simply tell the kernel "start running this thread right now"...you have to wait your turn
<headius>
so the more threaded fibers you have, the more that context switching becomes your bottleneck
<headius>
threaded fibers can't ever compete there...context switch for stack-swapping fiber is tiny and doesn't deschedule any threads
<kares>
fibers are used a lot with Celluloid as each actor method is basically handled in a Fiber
<headius>
it's used more like state...they have a particular thread get into an actor it can't complete and you need that thread back
<headius>
the code the fibers run is not thread-specific until the point it has to pause
<headius>
that's the use case that is hard to support with these big fat threaded fiberrs: call stack as implicit CPS state
<eregon>
right, so they yield on the message loop
<headius>
yes
<headius>
but I don't know if we're talking "too many" fibers or not
<headius>
my gut feeling is that it's hundreds, not thousands
<headius>
cremes: hey how many fibers does your celluloid thingy end up using?
<headius>
eregon: tbh, Enumerator#next is more of a worry...most people opting into Fiber know what they're getting into, and the conceptual cost usually keeps fiber counts under control
<headius>
plus you can't have 10000-free-floating fibers anyway in MRI...they'd still be bound to one logical thread of execution
<eregon>
In my Oz implementation on top of Truffle, we do need lightweight threads in like potentially thousands of them (the original impl can even handle >100k), so hopefully my Graal fork will help for that :)
<headius>
we really need VM-level support :-(
<headius>
I have not had the vocabulary to convince JVM peeps
<eregon>
let's push the JVM guys into that direction :)
<headius>
maybe we can get Lukas's patch out of mothballs
<eregon>
the simple coroutine patch is reasonably nice
<headius>
it wasn't that big...it's just all C++ JVM internals
<eregon>
that's what I reapplied to Graal, it's not very hard
<headius>
why does your solution need truffle at all?
<eregon>
generators are stackless in Python, isn't it?
<headius>
you can only yield from within the literal generator code
<headius>
so one frame, but that's fakeable
<eregon>
yeah exactly
<headius>
jython did it with trampolining too
<eregon>
like the future JS genertors, can be compiled to a big state machine, but not trivial to do
<headius>
right
<headius>
that's basically what we'd do if we made them trampoline through the whole stack
<headius>
our own little continuation-passing state machine
<headius>
blech
<eregon>
:D
<headius>
understandably we've never done it
<eregon>
so tiem to write a Ruby interpreter in Ruby CPS-style?^^
<eregon>
time*
<headius>
I modified JRuby's old interpreter to be CPS for a while in 2005
<headius>
fib(100k)!
<headius>
it was so slow
<headius>
well it was stackless...there wasn't much continuation-passing happening
<headius>
it could have supported fibers, unlimited continuations, all that stuff, at the cost of being unusably slow
<headius>
and it still breaks when you call through foreign code
<eregon>
I'm more looking to coroutines than all kind of crazy hacks like CPS and longjmp
<eregon>
more structured and less easy to shoot your foot but forgetting about exceptions, etc
<eregon>
by forgetting*
<headius>
yeah I kept hoping if I just kept making noise the JVM would get proper coroutines
<headius>
it didn't happen
skade has quit [Quit: Computer has gone to sleep.]
<cremes>
headius: my code probably spun up 40-50 actors and then called #future on most of them. If there’s a fiber per future then probably around 15k+ calls.
<headius>
I'm not sure when it spins up fibers for that
<headius>
did you get a peak thread count on JRuby?
<headius>
the only thing that really matters is *live* fibers that are yielded
<headius>
fibers that run to termination just go away
<headius>
abandoned fibers eventually go away
<headius>
cremes: we were just discussing next steps to get lightweight coroutines for JRuby and JRuby+Truffle
<chrisseaton>
cremes: Rubinius has just decided to use full threads again, hasn't it?
<headius>
wot?
<Antiarc>
cremes: I do a lot of stuff like that, the biggest performance concern is the thread pool that backs fibers, not the fibers themselves. I do it via celluloid - jruby at one point had some nasty pooling issues but those got fixed way back in the mid 1.7 days
<cremes>
chrisseaton: correct. the new Fiber implementation has every fiber backed by a pthread. reasoning was so that debuggers could see the stack and the OS could handle scheduling.
<headius>
you don't want the OS handling scheduling
<headius>
not for these
<headius>
brb
<cremes>
headius: I can’t speak for brixen but I think he’d say that fibers are an anti-pattern anyway, so don’t use them for perf-critical areas.
<cremes>
and don’t spin up 100k+ of them :)
<enebo>
cremes: so don’t be erlang
<cremes>
right, rubinius won’t be erlang
<cremes>
I’ll admit I don’t really understand the issues here and fibers give me a headache. I hate debugging fiber-based code which unfortunately means lots of headaches with celluloid.
<enebo>
I actually should not draw too close of a parallel to erlang. I think one reason erlang processes work so well is it is just a function and very little state
<Antiarc>
cremes: if you're just looking for a perf boost when using Celluloid heavily under JRuby, I wrote https://github.com/celluloid/celluloid-task-pooledfiber specifically for that purpose. It basically just maintains a fiber pool to avoid creation overhead all over the place. I haven't tested it since 1.7.11 days but it confers performance benefits even on MRI. Original PR here: https://github.com/celluloid/celluloid/pull/371
<Antiarc>
Rather I haven't tested the performance, I still use it :)
<cremes>
Antiarc: so funny but I just pushed a bunch of changes to that repository and branded a 0.2.0 release :)
<cremes>
it now works with master
<Antiarc>
Oh. Well, excellent then!
<Antiarc>
I'm still caffeinating, so please forgive me when I miss the obvious :)
<cremes>
Antiarc: I’m mentoring some GSoC students on celluloid work and ran into an issue where pooled fibers would be useful. Thanks for producing that nice add-on!
<cremes>
it had a little bit-rot but not much
<Antiarc>
Yeah, we're still on celluloid 0.16.x
<Antiarc>
So I hadn't touched it in a while
<Antiarc>
0.17 was a pretty big rewrite of a bunch of the internals that I'd painfully already debugged once, wasn't in a hurry to repeat that
<Antiarc>
Glad you found it useful!
<cremes>
Antiarc: heh, I understand about the rewrite. Once upon a time I grok’ed how celluloid worked but now with so many changes I am starting from square one again.
<cremes>
oh well
<Antiarc>
I keep having to put off the temptation to write my own Celluloid replacement
<GitHub9>
[jruby] chrisseaton pushed 1 new commit to truffle-head: https://git.io/vo7fk
<GitHub9>
jruby/truffle-head 211d31a Chris Seaton: [Truffle] Speculatively update to JVMCI JDK 0.16
<Antiarc>
I've got a hrorible chronic case of the Not Invented Heres
<cremes>
Antiarc: dont’ do it! help us make it better and more useful.
<headius>
or rather, don't spin up > ulimit fibers
<cremes>
@digitalextremist: join me in telling Antiarc to not rewrite celluloid :)
<headius>
which kinda kills their utility as a cooperatively scheduled microthread
<Antiarc>
Hehe, I'm not in danger of doing it right now
<Antiarc>
It's just something I keep having to dissuade myself from every time I get annoyed with celluloid :)
<headius>
plus, threaded fibers are much slower to switch...so welcome to our hell!
<cremes>
headius: mmmm, warm and toasty here
<headius>
hopefully we can get real fibers into JRuby soon...threaded fibers are NOT sufficient
<headius>
cremes: Enumerator#next too?
<cremes>
headius: presumably. i am not aware of any special casing that would make Enumerator use fibers differently.
<headius>
we talked above about how Enumerator#next is the sneakier one...it's perfectly valid to have 1k or 10k or 100k enumerators in flight
<headius>
if you can't do lightweight threads, that backs up *really* fast
<headius>
there's just so many problems with threaded fibers :-(
<cremes>
headius: this is where i leave the conversation to the runtime maintainers. i don’t know enough on the topic to be useful and just defer to your expertise.
<headius>
that conversation probably won't happen
<cremes>
heh, right. ‘nuff said on that.
<headius>
anyway...eregon: I'll have to play with your coroutine stuff
<headius>
if we can get it independent of Truffle I could implement lightweight threading in JRuby classic tomorrow
<headius>
I wouldn't bother with fallback...if you run on graal you can turn on true fibers
<headius>
it will explode spectacularly if you lie
<eregon>
I use the fallback for developing and being able to debug multiple fibers easily
<eregon>
headius: but what you would need is of course custom JDK build to reuse Lukas' native coroutines
<eregon>
that Graal+JVMCI is also a JDK build of course :)
camlow3__ has quit [Remote host closed the connection]
camlow325 has joined #jruby
<GitHub57>
[jruby] djberg96 opened issue #3981: Unable to resolve type uint32_t with FFI https://git.io/vo7wA
<GitHub88>
[jruby] chrisseaton pushed 1 new commit to master: https://git.io/vo7rZ
<GitHub88>
jruby/master 2cb1805 Chris Seaton: [Truffle] Store elapsed time for benchmark samples, for use in warmup graphs.
skade has joined #jruby
camlow32_ has joined #jruby
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #jruby
camlow325 has quit [Ping timeout: 240 seconds]
camlow32_ has quit [Remote host closed the connection]
tcrawley is now known as tcrawley-away
pawnbox has quit [Remote host closed the connection]
<GitHub172>
[jruby] chrisseaton pushed 1 new commit to master: https://git.io/vo7Pq
<GitHub172>
jruby/master bd52959 Chris Seaton: [Truffle] Use a custom JavaException instead of RuntimeException to get around checked exceptions until they're translated.
pawnbox has joined #jruby
skade has quit [Quit: Computer has gone to sleep.]
pawnbox has quit [Ping timeout: 244 seconds]
<GitHub158>
[jruby] chrisseaton pushed 1 new commit to truffle-head: https://git.io/vo7XW
<GitHub158>
jruby/truffle-head 302974a Chris Seaton: Merge branch 'master' into truffle-head
<sam2000>
Hi guys, I need your help please. I am using jruby with rails (puma server) and loading a jar library. Is there a way to make rails 'reload' the jar when it is changed please?
<sam2000>
It is because at the moment I am developing the jar library as well so I need to stop and start the rails server every time I update the jar. It would be much faster if it can just re-laod the jar file
camlow325 has joined #jruby
camlow325 has quit [Remote host closed the connection]
<sam2000_>
Hi, I am using jruby with rails. I am using a clojure library (jar file) and when it changes I stop and start the rails server to pick up the changes in the jar. Is there a way to reload the jar without restarting the rails server (puma)? it would be great if someone could help me
eregon2 has joined #jruby
camlow32_ has joined #jruby
camlow32_ has quit [Remote host closed the connection]
camlow325 has quit [Ping timeout: 250 seconds]
pawnbox has joined #jruby
<headius>
sam2000_: hmm...rails does its own restarting that leaves the current JRuby instance up...that complicates this a bit
<headius>
if JRuby itself shut down and restarted, even in the same JVM, that would work fine
camlow325 has joined #jruby
<sam2000_>
Thanks very much for replying, I only need this during development, so I was wondering if I just add a ‘require’ just before the call to the library would it work?
pawnbox has quit [Ping timeout: 246 seconds]
<sam2000_>
headius_:Thanks very much for replying, I only need this during development, so I was wondering if I just add a ‘require’ just before the call to the library would it work?
<headius>
sam2000_: probably not, unfortunately...the jars loaded into a given JRuby instance are all added to a global namespace
<headius>
Rails is sneaky about this because it actually deletes Ruby classes out of the global namespace, but we can't do that as easily with loaded classes
<headius>
it might be worth posting on JRuby mailing list...we have people doing interesting embedding things and they might have thoughts
<sam2000_>
headius_: Thanks again. I understand, So it looks like stopping and starting the server is the only way then?
eregon2 has quit [Quit: Page closed]
<headius>
unfortunately, it is for now :-(
<headius>
I could envision a future where you have a specific namespace you load classes from, and that namespace can get rebooted, but it's not in plans
<sam2000_>
headius_: I will try the mailing list. By the way, thanks for all your work in Jruby. I love clojure but when it comes to web apps rails is miles ahead of anything you can get in the clojure space. But Jruby lets me enjoy the best of both worlds
camlow325 has quit [Remote host closed the connection]
<headius>
well that's great to hear! If rebooting server is especially slow for you I'd love to do some analysis
<headius>
we've been looking for folks with apps that take a long time to boot, so we can investigate if there's easy ways to improve it
camlow325 has joined #jruby
prasunanand has quit [Remote host closed the connection]
lanceball is now known as lance|afk
tcrawley is now known as tcrawley-away
<sam2000_>
Just time the start time now. rails server takes about 25 seconds to start which I don’t mind really. Does it sound normal?
camlow325 has quit [Remote host closed the connection]
<headius>
sam2000_: rails 5 eliminates some double-booting that has doubled our startup
<headius>
it would be interesting to set JRUBY_OPTS=-Xdebug.loadService.timing=true and send us the output, if you get around to it
<headius>
we can get a rough idea how long files take to load and how much of boot is app versus just loading code
<sam2000_>
headius_: I will do that and send it to you.
<sam2000_>
headius_: By the way , an unrealted question, rvm doesn’t seem to know about jruby versions beyond 9.0.5.0? is this a known issue?
<sam2000_>
headius_: Maybe they are slow keep in sync with the latest jrubies. Just wondering if you might know anything about this
<havenwood>
sam2000_: To get at newer version metadata: rvm get head
<sam2000_>
havenwood_: Thanks verymuch havenwood_ , But is head stable enough you think?
<havenwood>
sam2000_: It has been more stable than stable in the sense that there's no active feature development so head just has bug fixes and metadata updates.
e_dub has quit [Quit: ZZZzzz…]
<havenwood>
sam2000_: There were plans for more frequent metadata only releases since it's in maintenance mode but it hasn't happened yet.
<havenwood>
sam2000_: head has 9.1.1.0 but if you give me a few I'll update it with 9.1.2.0
camlow325 has joined #jruby
<sam2000_>
havenwood_: Terrific, Thanks very much havenwood. When do you plan to update it to 9.1.2.0?
<havenwood>
sam2000_: doing it now
camlow325 has quit [Remote host closed the connection]
<sam2000_>
havenwood_: That is fast! great news! :)
<headius>
havenwood: thank you
<sam2000_>
havenwood_: By the way thanks very much for rvm. That really saves me from a lot of pain :)
<havenwood>
sam2000_: done!
<havenwood>
headius: you're welcome!
<sam2000_>
havenwood_: Thanks very much for your help!
<sam2000_>
headius_: Thanks verymuch to you too! I was able to chat with the authors of Jruby and rvm . This looks like a great place :)
camlow325 has joined #jruby
<GitHub165>
[jruby] chrisseaton pushed 2 new commits to master: https://git.io/vo7hF
<GitHub165>
jruby/master 3fa6a11 Chris Seaton: Revert "[Truffle] If you don't invalidate the $value global, it'll cause a de-opt storm in the PE tests."...
<GitHub165>
jruby/master 33d7d4d Chris Seaton: [Truffle] Much simpler solution to the problem with the global in PE.
sam2000_ has quit [Quit: sam2000_]
<headius>
sam2000: feel free to hang out any time
<sam2000>
headius_: Thanks very much
camlow325 has quit [Remote host closed the connection]
<fedenusy>
hi everyone! i have a .jar generated by warbler, and i'm trying to access its contents from a scala program. after adding the .jar to my classpath, i'm able to access warbler's JarMain object, but unable to access the compiled ruby classes. any tips on how i can interact with my ruby code in this situation?
<sam2000>
headius_: I am uaing rails 4.2.6. Would it still be useful if I send you the debug output you mentioned(JRUBY_OPTS stuff )? Or were you expecting rails 5? ( JRUBY_OPTS =-Xdebug.loadService.timing=true )
kphns has joined #jruby
<kphns>
Hello -- anyone have any hints for tracking down the source of the following exception? "org.jruby.ir.operands.UndefinedValue cannot be cast to org.jruby.RubyBasicObject"
<kphns>
this happens in both 9.0.4.0 and 9.0.5.0 with Java 8. I haven't tried the 9.1 series yet. The only mention of this I can find is about self-referential optional args, which is not relevant here; the ruby function that's causing this is something like: