<mbj>
snusnu: pree whitequark/ast but easily portable
<mbj>
snusnu: BTW if you return AST::Node instances its NOT an sexp
<mbj>
snusnu: so it would be Axiom::Relation#to_ast
<snusnu>
mbj: i'm currently trying to come up with a different/better AST format for describing the base relations, and i was for a minute thinking that we would benefit from having the virtual relations be represented as an AST too, which i guess, isn't correct after all
<snusnu>
mbj: yeah you're right, i wasn't precise
<snusnu>
mbj: i'd really appreciate a minute of your brainpower, checking my proposed ast format for base relations, deal?
<mbj>
snusnu: okay, this is tooo interesting to continue clients work.
<snusnu>
dkubb: i've incorporated low hanging fruits in your comments, thx for that! .. i will now go on to start building an AST evaluator that will build the axiom objects
<snusnu>
dkubb: the source AST format will likely change compared to the one currently given in the specs, but that's ok
<snusnu>
dkubb: mbj gave me a nice little intro on how to do the processing, so i'm confident that i will have something working later on today
cored has quit [Ping timeout: 265 seconds]
<dkubb>
nice
<snusnu>
dkubb: so that was just to keep you up to date ;) .. if you have some time tho, i'd like to discuss your thoughts on renaming wrapped/group tuple/relation valued attributes
<dkubb>
yeah, I'd like to brainstorm some of the items in our current todo list, to make sure it includes the main things, and then I'll do a write-up of each of them to explain them
<snusnu>
sweet, lemme have a quick look at it again
<mbj>
dkubb, snusnu: I'm back online in 2 hours or so.
mbj has quit [Quit: leaving]
<dkubb>
snusnu: yeah the main thing with the memoizable extraction is that I have to reintegrate it with adamantium and then have all the code that depends on adamantium continue to work
<dkubb>
snusnu: it seems that I removed a few things from memoizable that other code was depending on, so I need to add it back
<dkubb>
snusnu: we probably also need to audit some of the support directories. there's probably quite a few things that could be extracted and reused across the ecosystem
<dkubb>
snusnu: one example I highlighted is the options stuff in axiom. we have lots of cases where we have class level "attributes" where we define some value for
<snusnu>
dkubb: yeah re the options thing, i'm not a huge fan of that, most of the *rom* projects don't really need it currently (and i'm unlikely to establish a need for such a thing i guess, since i rarely find it useful)
<snusnu>
dkubb: of course that doesn't mean at all that i'm against abstracting it
<snusnu>
dkubb: i'm really speaking about the rom parts here, i totally see how it's useful to the design of other projects in our ecosystem
<dkubb>
snusnu: yeah, I thought so too, but then I started auditing axiom and saw things like class methods with "static" data which I can replace with this stuff
cored has quit [Ping timeout: 272 seconds]
<dkubb>
snusnu: although since I am freezing those classes in axiom-types, maybe it doesn't apply
<snusnu>
dkubb: yeah, i'd say that if it's useful in a few contexts, and these contexts would immediately benefit from a refactor to using that, then we should go for it
<snusnu>
dkubb: if only as a stop gap
<snusnu>
dkubb: so re the renaming in wrapped/grouped tuples/relations .. what are your thoughts on that?
<snusnu>
dkubb: it often happens that for joining, non-key attributes need to be renamed (say person.name and task.name)
<snusnu>
dkubb: so you need to rename them in order to get into a state where wrapping/grouping even makes sense
<snusnu>
dkubb: then you wrap/group
<snusnu>
dkubb: but then you actually want their original names again (especially for mapping into objects)
<snusnu>
dkubb: what i'm getting at i guess, is that there should be a way, to get wrapped/grouped values which have their "original" attribute names, while this of course has to be "undoable" for eventual unwrap/ungroup
<snusnu>
dkubb: i'm a bit wary of those parts, because they were causing substantial pain back then with dm-mapper
<snusnu>
dkubb: so my hope is that this time, we can get it "right" ;)
adkron has quit [Ping timeout: 252 seconds]
<dkubb>
hmm
<dkubb>
snusnu: if you're loading, say customers.join(orders), you need to disambiguate the order specific attributes from the customer ones.. you could define a prefix for the order attributes and make sure the mapper knows about that
<dkubb>
the "aggregate root" objects probably wouldn't need a prefix, although you could use one if you wanted to make it consistent
<dkubb>
I wonder if unwrapping and ungrouping needs to accept a prefix
<dkubb>
in axiom I mean
<dkubb>
because if you unwrap, you're going to clobber the same-named attributes in the original relation
<dkubb>
I also wonder if wrap/group should take an optional prefix too
<dkubb>
so that they would *remove* that prefix when wrapping/grouping so you'd get nice and clean names in the nested relation
<dkubb>
so wrap/group would remove the prefix specified from the attributes, and unwrap/ungroup would add the prefix to the attributes
dkubb|away has joined #rom-rb
<snusnu>
dkubb: giving it a quick thought, that sounds like a nice idea!
<dkubb|away>
another option is to default the prefix to the name of the nested relation
<snusnu>
dkubb: if both wrap/group and unwrap/ungroup were to accept a prefix, it could work almost transparently (apart from having to give the prefix)
<dkubb|away>
so relation.unwrap(:address) would be equivalent to relation.unwrap(:address, 'address')
<snusnu>
hmm .. that'd only work for base relations tho?
dkubb has quit [Ping timeout: 264 seconds]
dkubb|away is now known as dkubb
<snusnu>
oh wait, no relation names involved here, right?
<dkubb>
I dunno why it would only have to work for base relations
<snusnu>
the could be prefixed with the attribute name, no?
<dkubb>
yeah, we're talking about attribute names only
<snusnu>
right
<snusnu>
that'd be awesome i guess! also, i actually see no reason for allowing to pass an explicit prefix?
<snusnu>
couldn't it be handled transparently inside axiom?
<snusnu>
just always use the name used for wrap/group as prefix
<dkubb>
it might be possible, I'd have to see
<dkubb>
worst case I can just make the prefix default to '', and say that if you want to use it you need to be explicit
<dkubb>
which would be fine for ROM, since it would just always be specified when using those ops
<snusnu>
it might lead to a bit awkward "internal" names tho, as a wrapped attribute's attributes would be prefixed with a singular name mostly, whereas grouped ones with a plural name
<snusnu>
yeah
<dkubb>
I don't think internal names matters too much
<snusnu>
me neither, it just came to mind
<dkubb>
you could probably generate random strings inside ROM and use that if you wanted
<snusnu>
i certainly wouldn't want to involve an inflector for that :p
<snusnu>
hah yeah
<dkubb>
I mean, you *could*, I wouldn't want to
<dkubb>
I would probably try to keep the nested relation name 1:1 to the prefix
<dkubb>
it would be nicer when reading the SQL and trying to debug things
<snusnu>
by nested relation name you mean the name given to group, right?
<dkubb>
yeah
<snusnu>
right
<snusnu>
that surely sounds like a good plan to move forward
<dkubb>
hmm, one thing I realized just a sec ago, is that you should only be able to wrap attributes that are a superset of a key
<dkubb>
hmm, weird failure to see on a doc change
<snusnu>
heh
<snusnu>
so, what does it mean that you should only be able to wrap attributes that are a superset of a key .. i assume that applies for the *wrapped*/grouped relation?
<snusnu>
i guess the reason for this is, that otherwise you have no chance at identifying the tuple that needs to be updated after unwrap?
<dkubb>
actually, nevermind, I was wrong
<snusnu>
heh ok, so we're still fine with the prefix idea?
<snusnu>
dkubb: btw, do you think that it will eventually possible to ask "any" relation about its wrapped/grouped tuples/relations?
<snusnu>
dkubb: reason i'm asking, is that with having such info, we could probably generate mappers for wrapped/group relations automatically
<snusnu>
dkubb: that shouldn't be priority of course, but it came to mind
<dkubb>
snusnu: each relation attribute has a type, and the type will be Axiom::Relation or Axiom::Tuple depending on if it's grouped or wrapped
<dkubb>
snusnu: so it would be possible to introspect this now
<snusnu>
dkubb: here's a quick idea for mapping such "relations with relationships": http://pastie.org/8515265 .. notice how the mapper involving EV/EC needs to be stated explicitly, whereas the base relation mappers can be inferred easily
<snusnu>
dkubb: nice
<dkubb>
attributes can be relation or tuple types, in addition to the normal set of primitives
<snusnu>
yeah, i've seen that already, haven't connected the dots tho just yet, it seems ;)
<dkubb>
ahh that's nice
<dkubb>
we can infer the normal case where things are 1:1
<snusnu>
yeah
<dkubb>
in some ways we're allowing people to push things below the mapper so the mapper just sees relations
<dkubb>
where in fact they are views composed of other base relations
<snusnu>
yeah, to be precise, it only ever sees tuples, those could be tuple or relation valued tho
<snusnu>
ducktrap handles the transformations
<snusnu>
to be even more precise, it only ever sees tuples, but their *attributes* could be tuple or relation valued
<snusnu>
ducktrap can easily handle that, arbitrarily nested
<snusnu>
dkubb: btw note, that in the mapper builder spike xample i just sent, i explicitly made the connection to relations within an option hash .. no idea if that's gonna be the eventual api, but the idea behind it was, that mappers should be entirely possible to build *without* an axiom schema, thus without any relations to infer from
<snusnu>
in which case they'd expose the #map api, which would allow a typed or an untyped form
<snusnu>
also they'd expose #wrap/#group
<dkubb>
hmm, I think I'd probably need to see an example with more of the options to wrap my brain around it
kleech has quit [Remote host closed the connection]
<snusnu>
that's what ducktrap rom-mapper currently supports, it makes no sense to pass typing info when used with the rom stack, as the types are defined by the schema (only backend types are relevant for #dump, in case of #load, data coming from underlying relations is to be trusted)
<snusnu>
when used without relations under the hood, you'd still want to support type coercion powered by ducktrap tho .. the current api when giving types, is not yet ideal, as it only allows to give *one* type when it should leverage the bidirectional "nature" of ducktrap
<snusnu>
but i guess i'll postpone that until morpher is available, anyway, it's also not the "golden path" .. we should make it work with the rom stack first and foremost
<snusnu>
but not do anything that couples us too tightly to axiom
mbj has joined #rom-rb
<mbj>
dkubb, snusnu: back
<snusnu>
mbj: i just made my first evaluator/compiler spec pass: it builds an Axiom::Attribute.coerce([:id, Integer]) from s(:attribute, :id, Integer) # => Axiom::Attribute::Integer.new(:id)
<snusnu>
mbj: currently driven by Axiom::Schema::AST.evaluate(node, compiler = Compiler)
<snusnu>
mbj: so injecting a DDL generating compiler will be a breeze
<mbj>
snusnu: nice
<mbj>
snusnu: We could write an external ast emitter like this also.
<dkubb>
I believe twitter released this and I remember people freaking out because it wasn't a true bijection in some cases: https://github.com/twitter/bijection
<mbj>
dkubb: I remember you pointed me to that thing once.
<mbj>
dkubb: So what if my input format would only like to accept decimal notation, but not scientific?
<mbj>
dkubb: The same for floats.
<mbj>
It should be possible to whitelist coercion formats.
kleech has joined #rom-rb
kleech has quit [Ping timeout: 246 seconds]
<dkubb>
mbj: ahh I see
<mbj>
dkubb: scientific notation allows to "blow" up memory with some very small input.
<mbj>
dkubb: Remember ducktrap and friends are written for hostile input.
<mbj>
dkubb: Crashing a rails app relying on a virtus model coercing string params to hash would be very easy with 10e100000000000000
<mbj>
because a bignum with lots of memory would be allocated....
<dkubb>
I wonder if we should remove that
<dkubb>
maybe submit a ticket to coercible?
<mbj>
*did not tried, but I expect multiple vectors like this.
<mbj>
dkubb: So I'd prefer to have a coercer that fits within 32bit integer etc.
<mbj>
dkubb: We need to defend our apps like this.
<mbj>
I *love* ducktyping but I want to be very strict on the types I accept from the outside.
<mbj>
you have to be strict once. Read that erlang "do not programm defensively". They tell you to avoid defensive checks AFTER data entered the VM.
<mbj>
But you should be VERY defensive once it enters.
<dkubb>
mbj: it's probably better to specify a type and have the regexp inferred from that
<dkubb>
the type could say "expect numbers 1 to 100"
<mbj>
dkubb: good point +1
<mbj>
dkubb: But I'd only do structural validations.
<mbj>
Not domain validations.
<mbj>
Structural == types.
<dkubb>
axiom-types provides some mechnism to specify this
<dkubb>
I need to make it so that it uses functions to describe the constraints though, so they can be reflected on
<mbj>
So its okay for me to check for int32 first and later check in a domain-level validator for a typical human age.
<mbj>
ducktrap / morpher is for type validation, not domain validation.
<mbj>
I'll create shorthands for int32, int64 etc.
<mbj>
Will be very handy because DBs have hard limits also.
kleech has joined #rom-rb
<dkubb>
I kinda wish programming languages would get away from specifying int32, int64, etc. they should specify the constraints and allow the compiler to determine the optimal primitive
<mbj>
dkubb: I dont think we should abstract this to far.
<mbj>
dkubb: Register width in processors are fixed. And many database vendors must honor this.
<mbj>
dkubb: As we deal with databases in ROM we should honor this also.
<dkubb>
yeah, I mean, I know we have to do it because we need to match the databases
<mbj>
I'm totally okay with rubies Integer / Bignum promotion, but in this specific domain we must model the stuff we map to.
<dkubb>
I just mean in the general sense
<mbj>
dkubb: ahh
<mbj>
got you wrong +1 here ;)
<mbj>
But if you do arithmetic heavy stuff you must be able to nail down the native register widt.
<mbj>
Did more rust lately, it feels a littlebit like ruby.
<mbj>
I'm more happy with rust than any other compiled to static native language, currently.
<mbj>
Ported some embedded stuff with great success.
<dkubb>
when it matters, you should be able to specify the native type, but I think the type system should be able to either raise on a conflict or resolve it
<mbj>
dkubb: just testing how to crash coercible / virtus from params
<mbj>
dkubb: So I have a valid case when submitting ticket.
<dkubb>
like for example, if you have some input and you say, it must be between 0 and 100, and then you later go and use it somewhere else that says it needs to be an int32, it can raise an exception if one is not a subset of the other
vsorlov has quit [Ping timeout: 248 seconds]
kleech has quit [Remote host closed the connection]
snusnu has quit [Quit: Leaving.]
<dkubb>
mbj: did you see my comment on some of snusnu's code about moving the exception method inside the exception rather than having the caller being responsible for defining the error string?
<mbj>
dkubb: yeah
<dkubb>
mbj: I feel like this may be a better overall approach. the exception constructor accepts only what it needs to print the error message, and we get the error message coupled closer to the exception
<dkubb>
mbj: wdyt?
<mbj>
dkubb: I used the same strategy for some OSS lib already (dont remember)
<mbj>
dkubb: the message is an implementation detail.
<dkubb>
I realize it's a bit of a change from the norm, but it feels right
<mbj>
dkubb: And specing error generation strings really isnt fun.
<mbj>
dkubb: Centralizing all this specs is the far better option +1
<dkubb>
yeah, it does mean our code specs don't need to test the error message
<dkubb>
except maybe the specs for the exception class itself
<mbj>
dkubb: Maybe we should include Equalizer/Concord and spec the "inputs" for the exception
<dkubb>
but at least we're kind of pushing that nasty edge case out of the normal case, and into exception classes
<mbj>
where foo, bar are the inputs to the function raising the exceptions and the context to generate methods.
<mbj>
Dunno if rspecs raise_error compares via ==/eql?
<dkubb>
I think it would be enough to do expect { subject }.to raise_error(DomainError) unless foo or bar was optional
<mbj>
okay
<dkubb>
let's see what mutant says
<dkubb>
if mutant doesn't have a problem with it, then I'm fine with it.. as long as there are explicit specs for the exception instances somewhere
<dkubb>
I like that the normal code won't need to spec exception messages though
<dkubb>
I also think it makes it so we will define more precise exception objects when we want to vary the message
<dkubb>
rather than just using, say, ArgumentError everywhere we receive bad input to a method
<dkubb>
an ArgumentError subclass is fine of course
<dkubb>
and ArgumentError is fine too when it's a general kind of input error.. but when it's precise and something you want to report back tro the user, like maybe the user passed in unknown keys, then you can subclass it and use something more precise
<mbj>
heh, also we remove some of solnics concerns with this pattern ;)
<dkubb>
more like we push it somewhere else
<mbj>
He disliked he had to spec expectation messages.
<dkubb>
but I'm fine with that too
<dkubb>
sometimes you can't always remove complexity, but if you can push it somewhere out of the way then that's almost as good
snusnu has joined #rom-rb
<mbj>
snusnu: Pls read last 50 lines of backlog
<dkubb>
mutant will still require that you cover the expectation message in the error classes' specs
<mbj>
snusnu: IMHO a good pattern is described here, regarding exception messages.
<mbj>
dkubb: Totally okay. You spec it once for the whole app.
<dkubb>
I think snusnu is already on board with this, but it's good to make sure we have consensus
<snusnu>
yeah, i'm down with that
<mbj>
gonna go to sleep, cu
mbj has quit [Quit: leaving]
cored has quit [Ping timeout: 265 seconds]
cored has joined #rom-rb
cored has quit [Ping timeout: 248 seconds]
cored has joined #rom-rb
kleech has joined #rom-rb
kleech has quit [Remote host closed the connection]
kleech has joined #rom-rb
kleech has quit [Remote host closed the connection]
avdi has quit [Quit: Connection closed for inactivity]
kleech has joined #rom-rb
kleech has quit [Remote host closed the connection]