samphippen has quit [Read error: Connection reset by peer]
samphippen has joined #jruby
cremes has quit [Read error: No route to host]
cremes has joined #jruby
lance|afk is now known as lanceball
pawnbox_ has joined #jruby
jensnockert has joined #jruby
pawnbox has quit [Ping timeout: 240 seconds]
rsim has joined #jruby
enebo has joined #jruby
colinsurprenant has joined #jruby
jensnockert has quit [Ping timeout: 272 seconds]
tenderlove has quit [Ping timeout: 265 seconds]
drbobbeaty has quit [Read error: Connection reset by peer]
drbobbeaty has joined #jruby
subbu|afk is now known as subbu
tenderlove has joined #jruby
jensnockert has joined #jruby
jensnockert has quit [Ping timeout: 250 seconds]
rsim has quit [Quit: Leaving.]
rsim has joined #jruby
rsim has quit [Client Quit]
samphippen has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
pawnbox_ has quit [Remote host closed the connection]
jensnockert has quit [Ping timeout: 268 seconds]
pawnbox has joined #jruby
Liothen has quit [Ping timeout: 260 seconds]
mdedetrich has joined #jruby
Liothen has joined #jruby
jensnockert has joined #jruby
jensnockert has quit [Ping timeout: 244 seconds]
<nirvdrum>
enebo: I've now removed all usages of this 1.8 mode in ConvertBytes. But there are still some public methods using it. Kill them? Or do you think someone somewhere might be using that API?
<enebo>
nirvdrum: possibly just deprecate
<enebo>
nirvdrum: then it will continue to work for any native extensions which happen to be calling them
<enebo>
nirvdrum: At this point these APIs made it into our final release
<nirvdrum>
Yeah. It just means having this extra flag hanging around that doesn't do anything for JRuby so it'll never be tested and keeps triggering static analysis warnings.
<headius>
flag?
<nirvdrum>
Boolean field in ConvertBytes.
<nirvdrum>
Not a huge deal. I was just trying to tidy up and reduce cognitive load.
<headius>
what static analysis warnings are you talking about?
<nirvdrum>
Well, now is19 is always false from anything reachable.
<nirvdrum>
Sorry, it's always true.
<headius>
what tool is warning though?
<headius>
it's a private fields, remove it, replace all usages with equivalent of true and deprecate the constructor that takes it
<headius>
it looks like there's only one use of the field
<nirvdrum>
Oh, I was running the IntelliJ inspection.
jensnockert has joined #jruby
<nirvdrum>
headius: Correct. My question is how far does public API support reach? I accidentally changed the semantics of RubyNumeric.str2inum. But if that's a method that needs to be supported, it used the 1.8 semantics.
<nirvdrum>
I happen to think that was a bug, since every usage in JRuby using that method really needs 1.9.
<enebo>
I added an ass load of windows stat code over the weekend. Completely new impl which does not use windows compar POSIX stat method anymore
<headius>
heh, flag called "badcheck"
<enebo>
I am mildly nervous about it but it works with long path names and also UNC path names (and long UNC path names)
<headius>
so the concern is the bytelistToInum public API eh?
<enebo>
nirvdrum: you might know the answer to this….
<nirvdrum>
This stuff is all for deciding what to do with a number like "_35". 1.9 mode treats it as 0, 1.8 mode treats it as 35. badCheck fails with an exception regardless of the 19 flag.
<headius>
str2inum doesn't appear to be called by anything of ours, eh?
<nirvdrum>
There are two variants.
<headius>
oh, the non-flag one isn't called
<enebo>
nirvdrum: I had a struct which had a struct field but it was byvalue and not byreference and I could not figure out how that worked (although now I just realized one thing I did not try — nested POJO with nested struct fileds in it?)
mdedetrich has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<nirvdrum>
One is never called (similar deal with a variant of str2fnum).
jensnockert has quit [Ping timeout: 250 seconds]
<enebo>
nirvdrum: I can say from my oj porting that I called something for converting a string to a num
<headius>
MRI 2.2 appears to still have the badcheck version
<nirvdrum>
enebo: The @In and @Out annotations play a role in all that, too.
<nirvdrum>
headius: badCheck needs to stay.
<enebo>
nirvdrum: yeah but those annos are only for the args to the function right?
Liothen has joined #jruby
Liothen has quit [Changing host]
<headius>
rb_str_to_inum has it
<enebo>
nirvdrum: I cheated for now and just menually put those value struct fields into the top struct
<headius>
rb_cstr_to_inum has it
<enebo>
nirvdrum: I messed around trying to make it work for hours :|
<nirvdrum>
enebo: Right. But it'll treat the data flowing in if needed. E.g., if you give it a byte[], it'll create the reference wrapper I believe.
<enebo>
but it is a struct with struct fields
<nirvdrum>
Yeah. I don't know how deep it reaches. I was just suggesting it might have a hand. I don't know for certain.
<enebo>
FILE_DATA_INFO { TIMEDATE createTime … };
<enebo>
It seems this is so common I feel I must be missing something
<enebo>
but what I have works and I have a fixme
<nirvdrum>
enebo: I'm not entirely sure what the question or problem is :-)
<headius>
nirvdrum: I don't know if all our logic matches what's in MRI but they appear to still have the form with the flag and it does slightly different things
<nirvdrum>
headius: It's really a question of whether we need to retain this 1.8 compat for potential extensions.
<headius>
or more things at least
<nirvdrum>
The 1.8 stuff was supposed to die. This made it through because three Ruby methods implemented by JRuby were incorrectly using the old version.
<nirvdrum>
I fixed that.
<enebo>
nirvdrum: headius: It is likely they are wrong by using 1.8 version in an extension except for cases where you know the encoding is basically 7bit. Since there is still a use which is not broken I don’t think we should remove it
<enebo>
but if you changed to 1.9 and it is ascii then it should not matter and it should still work but the signrature itself should exit
<enebo>
exist
<headius>
nirvdrum: what I'm saying is that this flag doesn't seem to just denote 1.8 behavior
<headius>
it still propagates through MRI's version of this code and does things like hard error for invalid input
<nirvdrum>
headius: I think you're looking at the wrong flag.
djbkd has quit [Remote host closed the connection]
colinsurprenant has joined #jruby
djbkd has joined #jruby
djbkd has quit [Remote host closed the connection]
djbkd has joined #jruby
jensnockert has quit [Ping timeout: 240 seconds]
pawnbox has quit [Ping timeout: 250 seconds]
jensnockert has joined #jruby
subbu|lunch is now known as subbu
vifino has quit [Ping timeout: 246 seconds]
vifino has joined #jruby
pawnbox has joined #jruby
jensnockert has quit [Ping timeout: 272 seconds]
jensnockert has joined #jruby
<GitHub125>
[jruby] mkristian opened issue #3342: File.file?( "uri:classloader:/path/") can be true which is wrong http://git.io/vnRnq
jensnockert has quit [Ping timeout: 264 seconds]
ph has joined #jruby
jensnockert has joined #jruby
jensnockert has quit [Ping timeout: 264 seconds]
strax has joined #jruby
strax is now known as sebstrax
djbkd has quit [Remote host closed the connection]
djbkd has joined #jruby
havenwood has quit [Ping timeout: 264 seconds]
<colinsurprenant>
headius: following up on the ivars & voilatility - I am a bit on the fense for my current thread-safe ivars usage, I am now reliying on the implicit default volatility but if that default change back to non-volatile in a future release, it will break my code. thoughts?
jensnockert has joined #jruby
<colinsurprenant>
headius: obviously an option is to defensively wrap my ivars into a concurrent-ruby type today and whenever that default is changed it will be safe and eventually do the volatile :foo (or else) refactor at my pace when it lands
jensnockert has quit [Ping timeout: 246 seconds]
djbkd has quit [Remote host closed the connection]
djbkd has joined #jruby
rsim has quit [Quit: Leaving.]
jensnockert has joined #jruby
<headius>
colinsurprenant: well that's the thing
<headius>
I do not have a known piece of code that breaks with non-volatile ivars right now
<headius>
I think a first step that needs to happen immediately is to get a "volatile" gem out so you can depend on it and have the volatile function
<headius>
they might be no-ops for now but you'd be able to put them where you need them
<headius>
in a future version of JRuby they would set up that named ivar to use the volatile logic and ivars would otherwise be simple reads and writes
<headius>
when we can throw that switch is up for debate
<headius>
is the existing Synchronisation::Object in concurrent-ruby an option for you?
<headius>
it requires that you extend from that hierarchy, so that's a limitation
<headius>
I could flip the wiki back to saying "ivars aren't volatile, but here's how you do that"
ph has quit [Quit: ph]
<pitr-ch>
headius +1 for flip
ph- has joined #jruby
jensnockert has quit [Ping timeout: 250 seconds]
<pitr-ch>
in concurrent-ruby it can be done just by inheriting Synchronisation::Object and then creating volatile accessors with attr_volatile :name.
<pitr-ch>
headius from my perspective the gem is already out there. I wouldn't like to create another one just for volatile. However please let me know if you think c-r synchronisation needs improvements in any way.
vifino has quit [Ping timeout: 246 seconds]
cajone has joined #jruby
<colinsurprenant>
I was gonna say that the Synchronisation::Object with attr_volatile :name is pretty close to the volatile gem idea … do we need more? haviong to inherit the object makes it a bit less usable though
vifino has joined #jruby
jensnockert has joined #jruby
donV has quit [Quit: donV]
djbkd has quit [Remote host closed the connection]
havenwood has joined #jruby
djbkd has joined #jruby
djbkd has quit [Remote host closed the connection]
jensnockert has quit [Ping timeout: 240 seconds]
djbkd has joined #jruby
<colinsurprenant>
pitr-ch: is there a reason why Synchronisation::Object was not made into a module? it would be easier to use by including the module into the objest instead of using inheritance … ?
jensnockert has joined #jruby
jensnockert has quit [Ping timeout: 240 seconds]
djbkd has quit [Remote host closed the connection]
djbkd has joined #jruby
<headius>
colinsurprenant: I think everyone involved agrees it would be ideal to have Module#volatile
djbkd has quit [Remote host closed the connection]
<headius>
Synchronisation::Object is just what we have today
djbkd has joined #jruby
<chrisseaton>
I could do with some kind of shared planning document on this volatile discussion if people have the time - having trouble keeping up with the ideas!
<headius>
and I think we determined that it's possible to do Module#volatile in JRuby by just switching ivar table to use the volatile version of our logic
<headius>
chrisseaton: vast majority of the discussion is in gitter from the other day, but yeah, what sort of document would you recommend?
<chrisseaton>
just like a shared Google doc for now
<headius>
wiki page on concurrent-ruby maybe?
<headius>
something tied to other channels then
<headius>
we all know markdown
<colinsurprenant>
headius: I am putting up a small minimalist volatile gem I can use right-now, it will be a module and use AtomicReference basically like Synchronization::Object doesà
<headius>
colinsurprenant: ok great...maybe you want to make it under ruby-concurrency org with concurrent-ruby?
<GitHub96>
[jruby] chrisseaton pushed 1 new commit to master: http://git.io/vnR7a
<GitHub96>
jruby/master 954405a Chris Seaton: Copyright year.
<headius>
I believe I can give you rights to put it there
<colinsurprenant>
+1 wiki
pawnbox has quit [Remote host closed the connection]
<pitr-ch>
headius chrisseaton I am working on preparing documentation for the synchronisation layer, witch will also include informal ruby memory model description.
<headius>
colinsurprenant: I actually did `bundle gem volatile` about two hours ago and then got sidetracked
<headius>
I did rm -rf volatile about an hour later
<colinsurprenant>
lol
<colinsurprenant>
let me try see if I can get further :P
<headius>
colinsurprenant: I'm going to invite you as an owner in ruby-concurrent org, start up a volatile gem repo there
<pitr-ch>
colinsurprenant headius main reason why it is not an module right now is that some implementations may need helper instance variables or Java fields.
<headius>
we can make a flag for volatility in JRuby immediately and get the proper logic into volatile gem to work with that
<headius>
heh, I guess we only have owners in ruby-concurrency
<headius>
that's how we roll
<pitr-ch>
volatile using AtomicReference is not as efficient as what c-r has since it creates indirection and an object for each volatile field
<headius>
right
<headius>
even though I modified Atomic to use Unsafe rather than re-wrapping, it's still an extra layer
<headius>
btw, in Java 9 VarHandles will be our ticket to ride
<pitr-ch>
colinsurprenant I know inheriting is slightly less convenient, is it really a blocker for you?
<headius>
you'll be able to say "give me a method handle that represents volatile access to this field" without using unsafe
<pitr-ch>
headius yeah :)
<headius>
for array elements too
<headius>
pitr-ch: do you have a wiki page for memory model started yet?
<pitr-ch>
headius give me day or two to finish the MD. I am somewhere in the middle.
<headius>
ok
<colinsurprenant>
pitr-ch: it is… the class I need it for is already part of an inheritance hierarchy, it will be a lot easier to have it as a module
<pitr-ch>
For the first round of reviews i'll do probably a PR to make easier to comment though
<pitr-ch>
colinsurprenant composition does not work either I expect :/
<headius>
if you folks could add your thoughts to the redmine issue it would help me keep it fresh and at top of inboxes
<pitr-ch>
colinsurprenant that's only a volatile field with cas operations
<pitr-ch>
colinsurprenant just a volatile field is without the indirection
<pitr-ch>
volatile_with_cas will be without indirection too I hope in future
<pitr-ch>
colinsurprenant anyway I'll investigate if it could be turned into a module
<colinsurprenant>
ah yes I see
jensnockert has quit [Ping timeout: 256 seconds]
<headius>
module or Module#volatile are the same to me
<pitr-ch>
colinsurprenant another reason for class is to be able to provide safe publication of final instance variables initialised in costructor
<headius>
they'll both just flip some switches in our var table
<headius>
pitr-ch: but if they're marked volatile you have happens-before regardless of constructor behavior
<headius>
obviously a partially-intialized object could leak out but that can happen in Java too
<pitr-ch>
headius yeah this is ment for final ivars which do not have to be volatile
<headius>
ahh ok
jensnockert has joined #jruby
<headius>
ARM is another story altogether
<pitr-ch>
Yeah and I expect they start to leak to servers at some point
<pitr-ch>
yeah it's a noop, c-r is not trying to be correct just for x86
thedarkone2 has joined #jruby
<headius>
but we established volatile read on x86 is largely a no-op, right? defeats compiler optimzations, but otherwise a no-op
<pitr-ch>
though
<headius>
so the benefit of final is "volatile once" and accesses can be optimized by compiler
<headius>
right
<headius>
so if final requires a separate hierarchy, so be it
<headius>
so I guess my opinion on that is that we can do volatile without breaking any semantics of plain Ruby objects right now, but final is a concept foreign to Ruby
<headius>
volatile just modifies how the reads and writes to an ivar are done, while final has constructor publication, read-only outside of constructor, etc etc
<headius>
lots more semantics that would have to change
<headius>
a similar concept is the Clojure::Object I made
<pitr-ch>
headius hm, makes sense to separate
<headius>
where ivars were transactional and writes had to dosync { }
<headius>
foreign concept
<pitr-ch>
headius do you know about any examples how to define Ruby module form Java in JRuby ?
<headius>
just a class with static methods in it, for the most part
<headius>
module = true in JRubyMethod annotation
<pitr-ch>
headius it will have to provide the volatile var access and in future also cas operations
<headius>
well cas is a larger discussion
samphippen has joined #jruby
<headius>
but yes
<pitr-ch>
yeah, just mentioning it not to dig ourselves
jensnockert has quit [Ping timeout: 244 seconds]
<pitr-ch>
*I mean going in a bad direction
<headius>
attr accessor methods right now are just a type of DynamicMethod
<headius>
in JRuby
<headius>
atomic versions would just be DynamicMethod subclasses that know how to compareAndSwapObject
ph- has quit [Quit: ph-]
<headius>
see AttrReaderMethod and AttrWriterMethod in JRuby
<headius>
we'd just add AttrCASMethod, for example
<headius>
slow path invocation goes through that, fast path (e.g. indy) embeds appropriate Unsafe calls into call site binding
<headius>
and truffle does it however truffle does it, presumably with a CAS attr node
<pitr-ch>
yeah, there would be a node for it, or it would ruby code using some primitive like cas(receiver, :@ivar, expected, value)
<pitr-ch>
headius how do I get self in module method?
<chrisseaton>
atomic doesn't solve the += problem does it?
<headius>
you pass it
<headius>
chrisseaton: no
<headius>
that would require changing Ruby semantics
<chrisseaton>
i wonder if the name is going to give people false hope...
<chrisseaton>
'I have an atomic ivar so I can just +='
<headius>
pitr-ch: public static IRubyObject foo(ThreadContext context, IRubyObject self, ....)
<headius>
all static bound methods have to receive self before args
<GitHub118>
[jruby] chrisseaton pushed 1 new commit to master: http://git.io/vn0vC
<GitHub118>
jruby/master 6986805 Chris Seaton: Mention Truffle in COPYING.
<pitr-ch>
headius ah thanks
<pitr-ch>
yeah the people will have to know to use cas instead of +=
<chrisseaton>
at least it won't tear
<chrisseaton>
although no real common architectures tear anything at the moment I think
jensnockert has joined #jruby
<thedarkone2>
I did not occur to me, that we
<thedarkone2>
we've missed CAS
<thedarkone2>
I thought volatile :foo, implied also generating
<thedarkone2>
some cas_foo method
<thedarkone2>
just atomic attrs a border line useless for implementing concurrent algos
<GitHub18>
[jruby] chrisseaton pushed 1 new commit to truffle-head: http://git.io/vn0f0
<GitHub18>
jruby/truffle-head 08b752f Chris Seaton: Merge branch 'master' into truffle-head
<pitr-ch>
problem is that volatile we can do in the object, cas we cannot so far, hence it's separated in c-r and also in this discussion, at least that was my understading.
<headius>
now if 11539 doesn't get into Ruby 2.3, we could talk about trying to get both things into 2.4
<thedarkone2>
what do you mean? why would you need volatile if you have synchronize?
<headius>
you have to have volatile to ensure publication
<headius>
even with synchronized
<headius>
I actually asked bgoetz this exact question
<thedarkone2>
hm, I haven't though about it
<headius>
you can also do DCL with volatile
<headius>
safely
<pitr-ch>
headius you could have just normal fields protected by locks though
<headius>
to reduce that sync cost
<headius>
you can do a lot with just volatile and sync
<thedarkone2>
what pitc-ch says is what I'm thinking
<headius>
FTR I agree 100%, we need atomic too
<pitr-ch>
+1
<headius>
pitr-ch: but my understanding is that whether you're in a sync block or not, you still need them marked as volatile
<headius>
sync just replaces atomic operations
<headius>
it doesn't fix visibility
bbrowning is now known as bbrowning_away
<pitr-ch>
one more point about usefulness of library implementation: we can support volatile and cas in older JRuby and MRI until it's supported everywhere in few years
<headius>
right
<headius>
we will need a compat gem for all of this anyway
<thedarkone2>
yes, I still think it is better done a small gem
<headius>
so really, we *will* have volatile AND cas this year
<thedarkone2>
under c-r umbrella
jensnockert has quit [Ping timeout: 246 seconds]
<headius>
but ideally, for VM support, both should move into Ruby proper
<headius>
that's the longer tail
<headius>
for example, it will be hard for me to optimize c-r volatiles in my indy call sites
<headius>
because I can only talk about what I know in JRuby proper
<pitr-ch>
headius lock release makes visible all writes before the release to a thread which acquires the lock, no need for volatile. Or do I miss something ?
<headius>
once JRuby knows about Module#volatile, indy and IR will know about it
<headius>
pitr-ch: I don't think that's guaranteed to be true by JMM
<thedarkone2>
well, I think it should be our top priority, that whatever API is added, it should (in the future) be possible to implement it without any overhead
<headius>
it probably is true on x86, but for example ARM
<headius>
thedarkone2: agreed
<pitr-ch>
headius c-r will switch to whatever best impl available in JRuby fir volatiles based on version
<headius>
the other half of this is that 11539 is basically like adding proper volatile keyword semantics in Java
<headius>
AtomicReference and such are not part of th elanguage
<headius>
they're not in Java language spec
<chrisseaton>
yeah how do methods like that interact with the JLS?
<thedarkone2>
pitr-ch: I agree, I don't know why can't everything be done with only synchronize, but on the other hand I know better than to doubt Goetz ;)
<headius>
chrisseaton: I don't think there's anything about atomicity in JLS
<headius>
I have not looked though
<chrisseaton>
right, but the JLS defines what happens when you call a method - is there some big black hole somewhere where it say, 'oh and for some methods, like JNI, AtomicReference etc, they can do absolutely anything else not defined by the spec'
<thedarkone2>
I think there is no ARef in JLS
<headius>
chrisseaton: black hole sounds right
<headius>
so are we on same page?
<headius>
only "atomic" in JLS TOC talks about double tearing
<thedarkone2>
yes, unless the pdf is messed up
<thedarkone2>
there are not atomicrefs or atomicints in there
<headius>
thedarkone2: so you understand what I mean then
<thedarkone2>
so there is not cas?
<headius>
11539 is like what's in JLS
<headius>
how you actually do atomic on a given VM is currently brushed under the carpet of Atomic* classes
<headius>
it will probably be made more formal with VarHandles in 9
<headius>
or at least, I'd expect something more formal to come along with that
<lopex>
headius: and anything for arrays ?
<headius>
arrays too
<headius>
final arrays, volatile arrays
<headius>
64-bit striding views of bytes[]
<headius>
all of that stuff
<headius>
that's all in Sandoz's VarHandles
<lopex>
so the view would replace unsafe ?
<thedarkone2>
are VarHandles going into JLS? or they just wing it like with AtomicRef as a library?
lanceball is now known as lance|afk
<headius>
I don't know
<headius>
but I would theorize either JLS or JVMS will need to have something added
<headius>
[9/21/15, 4:30:55 PM] Charles Oliver Nutter: writes to a non-volatile field, even from within a synchronized block, do not get volatile write treatment, correct?
<headius>
[9/21/15, 4:31:23 PM] Brian Goetz: They do, but the VM may be able to combine the barriers.
<headius>
so there's a question of ordering of those writes within a given section if they're not volatile
cultureulterior1 has quit [Quit: cultureulterior1]
<headius>
versus happens-before guarantees when marked volatile
drbobbeaty has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
enebo has quit [Quit: enebo]
<headius>
so you can get a full publish with sync and non-volatile but you can't guarantee order
<chrisseaton>
order with other volatiles?
<pitr-ch>
headius yeah a,b vars in the synced block can happen ab or ba, but both have to happen before you acquire the lock so it should not matter.
<thedarkone2>
to that I reply: I'm gonna write myself a GVL and lets see if synchronize suffices!
<headius>
I'm confirming that interpretation
<headius>
chrisseaton: order with other non-volatiles
<headius>
the question was whether you could get away with just using non-volatile fields + synchronized
<headius>
and I think the answer is that you can only as long as the order of those non-volatile accesses is not important
<lopex>
no reorder with other volatiles were pre jmm afaik
jensnockert has quit [Ping timeout: 244 seconds]
<headius>
yeah it was something like that they fixe
<lopex>
a then .net fixed it as well :)
<headius>
yeah :-)
<lopex>
headius: remember that article that claimed properly constructed objects were free on x86 ?
samphippen has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<headius>
hmm no
<lopex>
they require a fence, but turns out it's free in that case