<yorickpeterse>
that way people can follow up on the status on the Psych issue
<mustmodify>
sure.
<mustmodify>
Just threw that together to demonstrate the issue.
<mustmodify>
I'm working on the ticket now
<mustmodify>
Hey were ivars introduced in rbx 2.2? I ask because this behavior isn't present in rbx 2.1
<mustmodify>
I'll check the changelog
<mustmodify>
it started with 2.2.2 :)
benlovell has quit [Ping timeout: 246 seconds]
tenderlove has joined #rubinius
<brixen>
mustmodify: it must be an interaction between psych and rbx
<brixen>
since it works on psych 2.0.6 and fails on psych 2.0.9 across many rbx versions
<brixen>
also, what does "were ivars introduced in rbx 2.2" even mean?
<mustmodify>
brixen: I'm sure that's true, since it doesn't happen in mri. But it's interesting that it doesn't happen in rbx-2.1.1
<brixen>
ivars are a Ruby language feature
<brixen>
I will debug this today, didn't have time yesterday
<mustmodify>
brixen: Ah, sorry. I haven't done enough research about ivars to know much about that feature, other than that it irritates me when the appear in hashes.
<|jemc|>
mustmodify: can you elaborate a bit on what you mean when you say "ivars"?
<|jemc|>
I wonder what ridiculous use case made that special serialization strategy make sense?
<|jemc|>
I'm trying to imagine how somebody trying to serialize a Hash with special "extra" data would want to do it by injecting hidden instance variables instead of just adding more keys to the Hash
max96at is now known as max96at|off
<|jemc|>
time to git blame our way back to the cause
<mustmodify>
|jemc|: There are some occult sects that perfer ivars in their hashes to keys.
<mustmodify>
|jemc|: because programming should be hard.
goyox86 has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<yorickpeterse>
The problem here is that for some reason this feature seems to break going from YAML -> Hash
<yorickpeterse>
at least in the case of bundler
<|jemc|>
yorickpeterse: well, the inability to deserialize properly may be *one* problem, but I consider psych creating an human-unreadable YAML file for a simple Hash to be the biggest problem
tenderlove has joined #rubinius
<|jemc|>
yorickpeterse: and now that I think about it - it's not surprising to me that it breaks coming back from YAML -> Hash when it's serialized like that
<|jemc|>
after all, I doubt Rubinius' version of Hash is meant to be created, then have its implementation instance variables filled by some other object
<|jemc|>
"I guess we could change it use composition instead of inheritance, but that would affect a lot of code paths for us, particularly in how we use these objects when communicating with a third party API."
<|jemc|>
translation: "I guess we could go back and fix our code to not rely on psych to do marshalling for us - but it's easier to change the behavior of a stdlib gem than for us to rewrite our own code correctly"
josh-k_ has quit [Remote host closed the connection]
goyox86 has joined #rubinius
<|jemc|>
err.. I suppose that's not an accurate translation - they'd still be using psych to marshal
<|jemc|>
more like: "we have a lot of bad code to fix - let's have a stdlib gem accomodate us instead"
diegoviola has joined #rubinius
goyox86 has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<|jemc|>
heh, the results of running psych tests on rbx:
Prathame_ has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<|jemc|>
but yeah, the failures don't look C-related - just some assumptions psych makes about serializing/deserializing ruby types
Prathame_ has joined #rubinius
Prathame_ has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<brixen>
|jemc|: we need to get psych working
<brixen>
|jemc|: whether that means patching psych or working around psych is not clear to me
<brixen>
I haven't had time yet to look at this
<|jemc|>
brixen: I've looked at it a bit, and it means a little of both
Bwild has joined #rubinius
benlovell has joined #rubinius
<yorickpeterse>
brixen: perhaps we need to adopt mirrors for Hash' internal state?
<yorickpeterse>
(heavens forbid using normal Ruby *gasp*)
<|jemc|>
I'm replying to tenderlove's comment on the psych issue now, although others are more than welcome to weigh in
<mustmodify>
yorickpeterse: My vote shouldn't count for anything, but I vote for fixing psych. Because exposing instance variables is odd. On the other hand, I guess it wouldn't be a hack to implement Hash#encode_with ...
<yorickpeterse>
I don't think it's unreasonable for Psych to _only_ export ivars for subclasses
<yorickpeterse>
so something like `if obj.is_a?(Hash) and obj.class != Hash`
<yorickpeterse>
but that also is a hack
tenderlove has quit [Remote host closed the connection]
tenderlove has joined #rubinius
<|jemc|>
blindly exporting/importing ivars to serialize/deserialize objects seems like a recipe for problems, in my mind
<yorickpeterse>
that too
<|jemc|>
it's just sad that the fools errand of arbitrary marshalling support in psych has broken the most basic use case ofdumping/parsing primitives
<mustmodify>
well, this issue seems to have exposed some raw nerves.
<mustmodify>
:(
<brixen>
mustmodify: are you getting push back with an attitude or something?
<mustmodify>
brixen: no I just think it's unfortunate that we're now in a "why does rubinius use instance variables instead of xyz" discussion instead of a "Hm... that does seem unfortunate, let's see what would be better" discussion.
<brixen>
oh, that's a very old discussion
<brixen>
and there are worse examples of it
<mustmodify>
brixen: I guess you guys are used to the "entrenched" issues.
<brixen>
I need to look at this carefully and will have a minute to do so shortly
<brixen>
it's not even so much entrenched as the assumption that "Ruby" is a thin veneer over C
<brixen>
or Java
<brixen>
and hence, any stupid thing that works in MRI is automatically Ruby
<brixen>
I say assumption vs entrenched because it has nothing to do with any reasonable consideration
<brixen>
the drawbacks of the Ruby is C assumption are really bad for Ruby
<brixen>
it's true if you push Matz et al on it, he defends it
<brixen>
like he did in the rubyconf 2014 chat with matz discussion
<brixen>
but there's no real conceptual / theoretic framework to it
<brixen>
"we hacked this language thing on C and that's pretty much what it is"
elia has quit [Quit: Computer has gone to sleep.]
<brixen>
it's that deep
<|jemc|>
headius: don't want to cloud the psych issue with unecessary noise, but to clarify real quick:
havenwood has quit [Remote host closed the connection]
<yorickpeterse>
but matz owns Ruby
<|jemc|>
"It would be necessary for any JVM-based Ruby to use Psych."
<|jemc|>
doesn't truffle start to add support for C extensions?
<yorickpeterse>
|jemc|: Truffle is, as far as I can tell, a thing on top of JRuby
<|jemc|>
(as an example of a JVM-based Ruby that wouldn't need a java implementation)
<yorickpeterse>
it's not yet the JRuby code, but chrisseaton probably knows more on this :)
<yorickpeterse>
s/code/core
<cpuguy83>
I thought Truffle was below JRuby/part of the JVM?
<|jemc|>
yorickpeterse: even if it doesn't exist/work yet, headius' comment suggested "any JVM-based Ruby" as if there were more than one leading me to believe that he was speaking of "any possible JVM-based Ruby"
<brixen>
one option we always have is to load a vendored version of psych and just ignore any attempt to load another version
<yorickpeterse>
hihi, this slidedeck is going to be good
<brixen>
not yet, I was referring to a need to understand why Rubinius implements Ruby classes in Ruby
<brixen>
any assumption that they are somehow special a la MRI's C wrappers is wrong
<|jemc|>
that's what I'm getting at in my comment
<brixen>
sweet
<|jemc|>
but I wanted you to take a look and correct me in the issue ticket if I'm misrepresenting the goals of rbx here
<|jemc|>
I don't really want to be taken as the spokesman for rbx in this comment - but it may be interpreted as such so I want to make sure I'm not saying the wrong thing here
<brixen>
|jemc|: PoLS applies only to matz, per matz himself, so don't use it in arguments ;)
<brixen>
a Ruby class uses instance variable, shocking!
<brixen>
one Ruby class uses instance variables that another does not use, equally shocking
<brixen>
Hash, String, etc are not special
<mustmodify>
brixen: PoLS may only apply to Matz but it's a worthy goal.
<brixen>
no generalization across them should be made
<brixen>
mustmodify: you need to define it then
<brixen>
a Ruby class uses instance variables that another does not use is not the least bit surprising
<brixen>
it's the very definition of the language
<mustmodify>
anyone who's like "Hey I hate PoLS" is essentially saying "Surprising people is fun!"
<mustmodify>
and they can go work in Redmond.
<brixen>
that's not what I'm saying
<|jemc|>
brixen: am I wrong to say we work to we work to "fix any widespread code (like psych) that assumes Arrays and Strings have an "empty" implementation."
<brixen>
Ruby classes behaving as Ruby classes is the least surprising thing
<mustmodify>
ding
<brixen>
|jemc|: we want to work to fix any assumptions that Ruby classes are not Ruby classes :)
<brixen>
what is the discussion of C-exts in that thread, I don't understand how that's relevant
<brixen>
and why do people not building rbx feel the need or the place to make proclamations on how rbx should be implemented
<brixen>
at this point, I'm more than happy to vendor psych and prevent any other version from loading
<brixen>
that's likely the simplest and least waste of time approach
benlovell has joined #rubinius
<yorickpeterse>
brixen: it's also the most painful in the long term
<|jemc|>
brixen: it's not relevant - the only reason I brought up the java extension was when headius was implying psych shouldn't add special code to work with rbx
<|jemc|>
and I also was never saying it should add special code for rbx
<brixen>
you should probably ignore any thing that headius says about anything related to rbx
<brixen>
that's just irrelevant noise that distracts from the main issue
<brixen>
which is that psych makes assumptions based on MRI's implementation of core classes in C
[spoiler] has quit [Quit: Leaving]
<brixen>
and should not make those assumptions, period
[spoiler] has joined #rubinius
<brixen>
there's really nothing to argue about
<brixen>
if psych refuses to *not* make those assumptions, we can 1) work around psych (ie by vendoring it) or 2) convince users of psych to work around psych
<brixen>
2 never works
<brixen>
finally, we can replace psych
<brixen>
the last has the highest benefit per effort, but has the longest lead time usually
<yorickpeterse>
I'm not going to write a replacement for Psych as well
<|jemc|>
well, if we get psych to not do the auto-marshalling by default (and make it an opt-in behavior) it is a "breaking change" for them, so we'll at the very least need to have a workaround until the next "breaking version" of psych
<brixen>
|jemc|: we really can't do that
<brixen>
if you set an ivar on a Hash instance in mri, you get the same serialization
<brixen>
in rbx, it's consistent
<brixen>
in mri, it's haphazard based on whether the user sets an ivar
<brixen>
I guarantee there is user code that sets ivars on Hash instances
<brixen>
and they'll probably be surprised now when they see their yaml output
<brixen>
this stupid crap has wasted a lot of people's time and effort
<brixen>
I need to get 2.5.0 out
<brixen>
so probably going to vendor 2.0.6 or 2.0.9 with this disabled
<brixen>
ugh, which requires more machinery
benlovell has quit [Ping timeout: 264 seconds]
<|jemc|>
brixen: by work around, I meant use syck or restrict to an earlier version of psych
<|jemc|>
also, give me my parsing instructions, and I'll replace psych for you :P
<brixen>
|jemc|: oh man, I am 1000% more motivated than my previous 10,000% :)
<|jemc|>
heh
<mustmodify>
I'm trying to figure out whether the question is "Are people who write classes and use instance variable obligated to hide them if they don't want them serialized" or "are instance variables actually a surface api?" (because my perception had been that message are the API and instance variables must be specifically exposed) or "is @headius an angry guy who needs a beer?"
<brixen>
at this point, the only person who would likely have an issue would be the OP of the original psych issue that broke this behavior
<brixen>
someone could reach out and ask if they use rbx
<brixen>
and offer to improve their deficient code, perhaps :)
<brixen>
mustmodify: could you please show me the Ruby facility that "hides" instance variables
<brixen>
or anyone
<brixen>
can anyone show me this
<brixen>
because I'm pretty sure there's no such thing as private instance variables
<brixen>
since they are already private lol
<mustmodify>
def instance_bariables; []; end
<brixen>
pretty much
<mustmodify>
def instance_variables; []; end
<brixen>
so, yeah, asking for a Ruby class to hide instance variables is just stupid
<brixen>
it's not a legitimate argument
<|jemc|>
brixen: well, this change did make psych behavior's more consistent - it's just in the wrong direction
<yorickpeterse>
we can do it the python way: @__foo
<yorickpeterse>
:D
<brixen>
and the fact that people suggest it just points out how terrible Ruby is
<brixen>
|jemc|: actually, yes, you're right
<|jemc|>
that is, they need to fix their current behavior for string and array
<yorickpeterse>
I'd argue that if people wanted to serialize ivars they should use Marshal, but then again I dislike YAML for this sort of stuff anyway
<brixen>
the *real* broken assumptions are everyone who uses YAML and assumes they get some exceptional rendering for Hash
<brixen>
and of course, YAML could so just go DIAF
<yorickpeterse>
The question is though, how does this break the bundler stuff?
<brixen>
it's a Hash, apparently
<mustmodify>
bundler dumps its (yaml) configuration and then can't read it.
<brixen>
serialized as a simple key-value set
<yorickpeterse>
But does that mean Psych can't convert YAML -> Hash in case of Rbx?
<yorickpeterse>
Or is bundler parsing the file using its own YAML parser?
<brixen>
ohh, it probably doesn't use YAML to read it
<brixen>
anyone want to confirm that?
<brixen>
I bet they read the file directly
<brixen>
"YAML is simple, let's just read the file"
<brixen>
"we can parse that with a regexp"
<brixen>
*collapses under a mountain of XML and dies"
<|jemc|>
yorickpeterse, brixen: I already gave my hypothesis
<brixen>
s/"/*/
<brixen>
|jemc|: sorry, where?
max96at is now known as max96at|off
<brixen>
I have 10 million things I'm tracking today
<|jemc|>
10:56:44 |jemc| | after all, I doubt Rubinius' version of Hash is meant to be created, then have its implementation instance variables filled by some other object
<|jemc|>
specifically, doing things like Rubinius::Tuple.allocate inside the Hasah are probably dumb
<|jemc|>
*Hash
<brixen>
name *any* Ruby class who's implementation's instance variables should be filled by some other object?
<|jemc|>
brixen: exactly
<brixen>
yeah, just Ruby
<brixen>
like, Ruby
<brixen>
why is this so hard
<brixen>
"Rubyists"
<brixen>
"you have **instance variables** on your objects, what the total fuck?!"
<|jemc|>
like I was saying, if you need to marshal an object - define a marshalling scheme
<brixen>
"On the 7th day, Matz the lord decreed that Hash shall have not the instance variables that Ruby hath, and the adder of instance variables to Hash shall be smitten, and their ashes strewn to the seven directions"
amsi has joined #rubinius
<|jemc|>
brixen: well, maybe we should check the specification of the language
<|jemc|>
oh wait...
<brixen>
|jemc|: a marshaling scheme, you mean, like Marshal? :)
<yorickpeterse>
|jemc|: surely we can buy the ISO spec for $200
<|jemc|>
yorickpeterse: surely this detail would be covered there
<brixen>
|jemc|: I'm sure you've seen JavaScripts falsey/truthy table thing?
<yorickpeterse>
brixen: well, Bundler at least uses a regexp for parsing ~/.bundle/config
<brixen>
|jemc|: be careful how much faith you put in a specification :)
<brixen>
that's reading the file and scanning it with a regexp, no?
* brixen
fully admits that he could be entering a phase of not even understanding Ruby code
<yorickpeterse>
brixen: Yes, but I want to be 100% sure
<yorickpeterse>
in case it's only used for $HOME/.bundle/config
<yorickpeterse>
and not $PWD/.bundle/config
<brixen>
man, you can *write* YAML by hand, you don't effen *read* YAML with a regex
<brixen>
good god
<[spoiler]>
I can't fathom why they're doing it, though.
<[spoiler]>
Unless it's some "performance bullshit"
goyox86 has joined #rubinius
<|jemc|>
yorickpeterse: at least with python they had the guts to purge the cruft in 2 to 3
<|jemc|>
you can't change ruby to make it better without breaking a lot of applications a la 2 to 3
<brixen>
|jemc|: sure you can :)
<[spoiler]>
|jemc|: and that's also the reason why hardly anyone [of my developer colleagues] switched to 3
<[spoiler]>
Ngl, I was so excited for rbxx because I felt like it'd ditch all the messy Tuny parts
<[spoiler]>
s/Tuny/Ruby
tenderlove has joined #rubinius
<brixen>
heh Tuny
<|jemc|>
[spoiler]: for a second I thought you were using Tuny as a pejorative term for Ruby :P
<[spoiler]>
Hahaha
<brixen>
ok, so there is no way we can allow the psych 2.0.9 behavior
<brixen>
probably 99.999% of YAML files out there include Hashes
<brixen>
and people expect to see the simple Hash representation
<brixen>
so 1) bundle psych and disallow loading it, 2) class Hash; def instance_variables; []; end; end, 3) burn Ruby to the ground and go herd goats in Peru
<brixen>
I'm open to votes here
<yorickpeterse>
I'd see if we can persuade tenderlove to revert the commit that changed this
<|jemc|>
mustmodify: I think you double posted by accident on the issue
<yorickpeterse>
since it was meant for a _subclass_ of Hash for a person's own use case
<brixen>
haw long do we wait for that?
tenderlove has quit [Ping timeout: 276 seconds]
<|jemc|>
yorickpeterse: but he's tenderlove is right to try to make his library more consistent - and to remain consistent he needs to make the string and other primitive behavior opt-in as well, which is a breaking change for crazy users that depend on this behavior
<|jemc|>
only reverting the commit that caused us to notice this is a hack only
<brixen>
pretty sure I need to add a -Xruby flag that gives you *Ruby* behavior
<brixen>
obviously, defaults to *off*, which gives you omg-wtf-MRI behavior
<|jemc|>
brixen: I'm saying no instance variables sucking by default
<brixen>
anyone want to open an MRI ticket for adding #private_instance_variables method? :)
<brixen>
they can hang out with private constants and have a private party
djinni has quit [Ping timeout: 244 seconds]
yipdw has quit [Quit: No Ping reply in 180 seconds.]
Evan_ has quit [*.net *.split]
locks has quit [*.net *.split]
clauswitt has quit [*.net *.split]
brixen has quit [*.net *.split]
saline has quit [*.net *.split]
ahkurtz has quit [*.net *.split]
elia has quit [*.net *.split]
imajes has quit [*.net *.split]
yxhuvud has quit [*.net *.split]
bakkdoor has quit [*.net *.split]
mqt has quit [*.net *.split]
erdic has quit [*.net *.split]
Liothen has quit [*.net *.split]
_ko10 has quit [*.net *.split]
tarcieri has quit [*.net *.split]
jeremyevans has quit [*.net *.split]
RealMarc has quit [*.net *.split]
Guest85414______ has quit [*.net *.split]
stass has quit [*.net *.split]
Dyrim has quit [*.net *.split]
justinmcp_ has quit [*.net *.split]
cyndis has quit [*.net *.split]
stormwind has quit [*.net *.split]
DireFog has quit [*.net *.split]
dmilith has quit [*.net *.split]
goyox86 has quit [*.net *.split]
nirvdrum has quit [*.net *.split]
eregon has quit [*.net *.split]
bennyklotz has quit [*.net *.split]
yorickpeterse has quit [*.net *.split]
|jemc-bot| has quit [*.net *.split]
mpapis has quit [*.net *.split]
blowmage has quit [*.net *.split]
max96at|off has quit [*.net *.split]
cremes has quit [*.net *.split]
cezarsa has quit [*.net *.split]
cpuguy83 has quit [*.net *.split]
coffeejunk has quit [*.net *.split]
pietr0 has quit [*.net *.split]
kagaro has quit [*.net *.split]
mrb_bk has quit [*.net *.split]
lbianc has quit [*.net *.split]
meh`_ has quit [*.net *.split]
Caius has quit [*.net *.split]
unreal has quit [*.net *.split]
pd has quit [*.net *.split]
heroux has quit [*.net *.split]
yopp has quit [*.net *.split]
havenwood has quit [*.net *.split]
dreinull75 has quit [*.net *.split]
guilleiguaran_ has quit [*.net *.split]
stormbre1 has quit [*.net *.split]
diegoviola has quit [*.net *.split]
|jemc| has quit [*.net *.split]
andrewstewart has quit [*.net *.split]
dlackty__ has quit [*.net *.split]
amsi has quit [*.net *.split]
chrisseaton has quit [*.net *.split]
lopex has quit [*.net *.split]
cypher23 has quit [*.net *.split]
LTe has quit [*.net *.split]
Spakman has quit [*.net *.split]
|Blaze| has quit [*.net *.split]
jc00ke has quit [*.net *.split]
[spoiler] has quit [*.net *.split]
pwh has quit [*.net *.split]
atambo has quit [*.net *.split]
mustmodify has quit [*.net *.split]
sbryant has quit [*.net *.split]
machty has quit [Max SendQ exceeded]
jeregrine has quit [Max SendQ exceeded]
amsi has joined #rubinius
cypher23 has joined #rubinius
flavorjones has quit [Ping timeout: 244 seconds]
dbussink has quit [Ping timeout: 244 seconds]
dbussink has joined #rubinius
LTe has joined #rubinius
imajes has joined #rubinius
jeregrine has joined #rubinius
dlackty__ has joined #rubinius
|jemc| has joined #rubinius
cezarsa has joined #rubinius
cpuguy83 has joined #rubinius
pietr0_ has joined #rubinius
stormwin1 has joined #rubinius
djinni has joined #rubinius
andrewstewart has joined #rubinius
jeremyevans has joined #rubinius
|Blaze| has joined #rubinius
coffeejunk has joined #rubinius
machty has joined #rubinius
bakkdoor has joined #rubinius
Liothen has joined #rubinius
<brixen>
yorickpeterse: be sure to ask headius why he insists on pontificating about decisions Rubinius should make for implementation
Spakman has joined #rubinius
flavorjones has joined #rubinius
sbryant has joined #rubinius
chrisseaton has joined #rubinius
yipdw_ has joined #rubinius
jc00ke_ has joined #rubinius
lopex has joined #rubinius
<brixen>
unbelievable
machty has quit [Max SendQ exceeded]
Liothen has quit [Changing host]
Liothen has joined #rubinius
<brixen>
yorickpeterse: I commented on the issue
<brixen>
I'm going to proceed with bundling psych and disallowing loading any other version