<coop-cooper>
Hey guys, I've been watching the ROM project for awhile now and I'd like to contribute. I have seen the Roadmap on the website but I was wondering if there was a more detailed "todo" list that I could attempt to pick at?
coop-cooper has quit [Ping timeout: 252 seconds]
coop-cooper has joined #rom-rb
coop-cooper has quit [Ping timeout: 245 seconds]
snusnu1 has joined #rom-rb
snusnu1 has quit [Read error: Connection reset by peer]
coop-cooper has joined #rom-rb
coop-cooper has quit [Ping timeout: 252 seconds]
coop-cooper has joined #rom-rb
coop-cooper has quit [Ping timeout: 245 seconds]
mbj has joined #rom-rb
coop-cooper has joined #rom-rb
mbj has quit [Ping timeout: 272 seconds]
coop-cooper has quit [Ping timeout: 240 seconds]
coop-cooper has joined #rom-rb
coop-cooper has quit [Ping timeout: 272 seconds]
coop-cooper has joined #rom-rb
mbj has joined #rom-rb
coop-cooper has quit [Ping timeout: 272 seconds]
postmodern has quit [Quit: Leaving]
snusnu1 has joined #rom-rb
mbj has quit [Ping timeout: 246 seconds]
mbj has joined #rom-rb
jfredett-w has joined #rom-rb
jfredett-w1 has quit [Ping timeout: 240 seconds]
lgierth has joined #rom-rb
snusnu1 has quit [Ping timeout: 246 seconds]
snusnu1 has joined #rom-rb
Gibheer has quit [Ping timeout: 245 seconds]
lgierth has quit [Quit: Ex-Chat]
Gibheer has joined #rom-rb
snusnu1 has quit [Quit: Leaving.]
<dkubb>
mbj: I'm nearing 1:1 feature parity with axiom-sql-generator. I was thinking about replacing the string concat with sql.rb and see if I can get all the specs to pass
<dkubb>
mbj: then I'll probably change it to try to create inner joins when possible instead of nested subqueries
<dkubb>
there's also a few other sql specific optimizations I can do
<dkubb>
like transforming: col = 1 OR col = 2 OR col = 3 into col IN (1, 2, 3)
<mbj>
Yeah
<mbj>
dkubb: AST based transforms are many times more easy than optimizing the string concatenator.
<mbj>
Also the SQL gem will be very usefull for other projects!
<dkubb>
yeah
<dkubb>
I'd love to work in the parser soon
<dkubb>
I think it will provide some extra tension that'll help solidify the ast structure
<dkubb>
right now it's basically optimized for generation
<dkubb>
but I want to make sure it works well for parsing
<mbj>
Yeah
<dkubb>
plus when I work on the fuzzer, that'll add constraints from another dimension
<mbj>
Optimizing down the number of nodes is a good strategy to ease parsing.
<mbj>
I think you'll remove the "named for positional signalling" again.
<mbj>
But TRY IT OUT FOR YOURSELF!
<mbj>
:D
<dkubb>
:)
<dkubb>
yeah
<dkubb>
I think positional is fine when the position is clear
<dkubb>
anything "functional" in nature would be fine
<dkubb>
like s(:and, node1, node2)
<dkubb>
but a SELECT query doesn't really fit that mold too well
<dkubb>
it's like a huge function with too many optional parameters
<mbj>
I think this is a misdesign of SQL, having such a powerful SELECT node :D
<mbj>
BTW both unparser and mutant have AST-processing backreferences.
<mbj>
I try to reduce their use.
<dkubb>
I'll check that out
<mbj>
IMHO a good designed AST / Language will try to reduce the need for backreferences.
<dkubb>
I'm still trying to decide if the parent should know about every possible child, or if the child should know what context to use depending on the parent
<mbj>
I tried both approaches.
<dkubb>
the problem with SQL is that some nodes can appear nearly anywhere
<mbj>
Mutant / Unparser both are hybrid in this sense.
<dkubb>
so every parent would have to know about every possible child
<mbj>
First go for correctness.
<dkubb>
I think it's going to be messy no matter what because SQL is messy
<mbj>
Emitting superflownous parentheses is perfectly fine.
<dkubb>
yeah, that's important to me
<dkubb>
I want the emitted SQL to be as close to what I would write by hand
<mbj>
In unparser foo << bar * baz gets emitted as foo << (bar * baz)
<dkubb>
just like unparser
<mbj>
dkubb: Do iterations!
<dkubb>
well, yeah, correctness will be #1
<mbj>
Dont try to write the first sql unparser with the #2 target in mind.
<mbj>
Get it correct than make it pretty.
<mbj>
I tried the same with unparser, and got far. With this strategy.
<dkubb>
yeah, I will
<dkubb>
I'll make sure the tests support me
<dkubb>
fuzzing will be a big part of this
<dkubb>
I need to get the parser working though
<dkubb>
so I can do round-trip tests
<mbj>
dkubb: I still need to do fuzzing for unparser.
<dkubb>
fuzzing will probably uncover lots of bugs in parser
<dkubb>
when I fuzzed axiom-sql-generator I found more bugs in the parsing of the SQL than I did in the generation
<mbj>
I know whitequark regulary tests parser against the rubygems corpus.
<dkubb>
yeah, that's a good idea
<mbj>
BUT he cannot verify correctness with this technique.
<mbj>
Only parser + unparser could be used for verification.
<mbj>
If you can execute the tests after unparsing.
<mbj>
Only problem are __LINE__ references.
<mbj>
But these could be emitted hardcoded.
<dkubb>
I wonder if you could use a markov chain to do the fuzzing
<mbj>
Good idea.
<mbj>
Have to try.
<mbj>
Additional idea: I wanna have a set of predicates per node that define if a node is valid.
<mbj>
For example s(:send, nil, nil, nil) is NOT valid.
<mbj>
s(:send, :foo, :baz) is valid ":foo.baz"
<mbj>
s(:send, :foo, :baz) is valid "foo.baz" * sorry
<mbj>
nah
<mbj>
both are invalid :D
<dkubb>
I still wonder if a lightweight "Send" object that has a #to_ast method that returns the ast might be better, then the validity checking can be encpasulated in the object
<mbj>
s(:send, nil, :baz) this one is valid.
<mbj>
I'm talking about our generic fuzzer.
<mbj>
The fuzzer must have some predicates that allow him to make sure no invalid AST gets generated.
<mbj>
Sure we can always add dedicated fuzzer objects per type. (mutant mutation emitter like).
<mbj>
But make we can write a generic fuzzer if you have predicates on the nodes.
<dkubb>
a markov chain would define the range allowed values for each node
<dkubb>
*range of
<mbj>
Yeah, a markov chain is the inverse of my predicates on nodes idea.
<mbj>
Or maybe the same :D
<dkubb>
so you start at the root, then randomly pick a node, then randomly pick some valid values for that node
<dkubb>
you could have some limits on the depth of the tree
<dkubb>
for any given node, other nodes could be valid arguments, so it could go on infinitely without some limit
<mbj>
I'll do a PR soon. Will help me to simplify unparser / mutant.
<dkubb>
ahh, that's cool
<mbj>
to provide a better selfcare.
<mbj>
s/selfcare/selftest/
<dkubb>
sql.rb should provide a list of valid node types
<mbj>
yeah
<dkubb>
simply by looking to see what names are registered
<mbj>
exactly
<dkubb>
btw, I would usually not have a constant be part of the public interface.. instead of Unparser::Emitter::REGISTRY.key?(type) you could have something like: Unparser::Emitter.handler(type)
<mbj>
yeah
<dkubb>
I've found I much prefer having a method that returns the handler, as opposed to having a key test
<mbj>
Sure.
<dkubb>
it allows me, when necessary, to create objects on-the-fly for handling
<mbj>
It was just easier to explain this way.
<dkubb>
instead of always relying on static handlers
<dkubb>
I did this in axiom-types and it simplified some stuff
<mbj>
For example I'd love to see Parser::CurrentRuby.node_types => Set.new(:lvar, :ivar, .... )
<dkubb>
that would be really nice
<mbj>
Parser::CurrentRuby.parse is the public interface.
<dkubb>
maybe your PR should include that
<mbj>
Dunno, making up a list of all ruby version specific nodes is to much for my OSS time frame.
<dkubb>
heh
<mbj>
1.8 has :not
<mbj>
1.9 not has :not
<mbj>
etc.
<dkubb>
just get ahold of whitequarks framework for testing rubygems
<dkubb>
then have it run through the gems and write out the nodes, per ruby type, to a file
<mbj>
good idea.
<dkubb>
then it's a matter of letting it run
<mbj>
But this will take longer than Parser::Meta::NODE_TYPES
<mbj>
And I need this to relihable close that :until bug in mutant :D
<mbj>
I dislike to close bugs without solving the infrastructure problems behind it.
<mbj>
For mutant I'll just add a blacklist of the 1.8 nodes when interpreting Parser::Meta::NODE_TYPES
<mbj>
dkubb: loops controlled by until and while could be named as: conditional loop? (searching the name for the Mutator::Node subclass for both).