<mbj>
IMHO a rubocop authors should have done some beta releases with newer deps.
<mbj>
BTW bundlers errors are easier to debug with:
<mbj>
DEBUG_RESOLVER=1 bundle foo
<jessekempf>
Ok, I'll track this down.
<jessekempf>
I'll see if I can hammer out all the dep conflicts and I'll let you know what I find later
<mbj>
just use that rubocop and use bundle exec
<mbj>
your problems will be gone.
<jessekempf>
I appreciate the help =)
<mbj>
Most likely mutant fails to load rspec
<mbj>
silently!
<mbj>
This causes the kill for to exit nonzero
<mbj>
yields false positives for evil mutations, but the noop sanity check saved us here.
<mbj>
noop basically injects the unmodified mutant
<mbj>
s/mutant/subject/
<mbj>
falling asleep, cu
<jessekempf>
kk
mbj has quit [Quit: leaving]
<dkubb>
O
<dkubb>
I'll be around a bit later tonight to help if you need it jessekempf
<dkubb>
probably 9 or 10 PM PST
<jessekempf>
dkubb: thanks. It looks like I might need it =/
<jessekempf>
bbiaw
jessekempf has quit [Quit: Leaving.]
jessekempf has joined #rom-rb
jessekempf has quit [Quit: Leaving.]
jessekempf has joined #rom-rb
<dkubb>
jessekempf: I just pushed a change to mutant on github that doesn't swallow exceptions, so maybe it'll report the issue rather than being a noop failure
<dkubb>
I'll test it against mutant-breaker in a moment, but I wanted to let you know first
<jessekempf>
dkubb: kk
<dkubb>
in either case, the logic we had there was catching too many exceptions. even if we decide to do it later, we can't be so broad with what we're catching
<jessekempf>
dkubb: I've got my own hacked version I'm working against. It's continuing to fail, and Rspec::Core::Runner isn't sticking anything in the StringIO it's being passed.
<dkubb>
oh I see
<dkubb>
jessekempf: did you merge the change I sent into your hacked version?
<dkubb>
what version of ruby are you using? I've been testing with ruby 2.0.0p247
<dkubb>
and running the spec normally does not produce this error?
<jessekempf>
On my laptop, 1.9.3-p125…which is a little old. My team uses 1.9.3-p374 in production. The company uses 1.9.2 for most of its stuff, but is upgrading to 1.9.3.
<jessekempf>
Correct, running the spec normally doesn't produce this error.
<dkubb>
interesting. lemme install that version and see if it's something platform specific
<dkubb>
I do know 1.9.2 has some really weird json problems
<jessekempf>
My guess is it's an old version of JSON
<dkubb>
like it requires the json gem for some things, because the built-in is so old
<dkubb>
:)
<jessekempf>
jesse.kempf:~/workspace/account-reifier/ddiflib-mutant2 [git: master] $ gem list | grep json
<jessekempf>
I don't know why you would do this, but between some ancient version of JSON and the modern one, they decided that JSON.load('') would return nil instead of an exception.
<dkubb>
hmm
<dkubb>
since 1.9.3-p125 will take a bit to install, I have one theory to test
<dkubb>
I noticed mutant isn't in the Gemfile for that repro
<dkubb>
which means it's probably using the mutant in the global path to run the specs
<dkubb>
which would also use the deps from the global gem path, not the stuff in your Gemfile
<jessekempf>
I added mutant to my gemspec
<dkubb>
so maybe if you added mutant, and then ran it with "bundle exec mutant ..."
<dkubb>
oh hmm, lemme check I must've missed it
<jessekempf>
That code's not up on github yet =)
<jessekempf>
er, =(
<dkubb>
ahh ok
<dkubb>
lemme know when you push it
<jessekempf>
working on it
<jessekempf>
Updated.
<dkubb>
fwiw, I didn't write mutant but I use it in oss and work projects and it's awesome
<dkubb>
it helps me find places I missed writing specs for
<dkubb>
and it helps find bugs earlier in some cases
<jessekempf>
We're working to the point where all code has to pass all rspecs, be clean according to rubocop, and pass reek without warnings before it's ready for a code review.
<jessekempf>
dkubb: sure
<dkubb>
that's a good approach
<dkubb>
I've done that on other teams and it reduces code review time, because it takes care of some of the low hanging fruit
<dkubb>
I hate reviewing code that is misformatted, and I have to send it back and wait for it to be updated before I can do a "real" review
<dkubb>
well, in truth I do a full pass anyway, but it always means I need to do two passes
<dkubb>
which sucks, because if those devs were using rubocop then at least most of the common stuff would've been fixed before I even saw it
<jessekempf>
yeah, you also get so much time on a CR before you make your CR agony quota and stop caring.
<jessekempf>
Gibheer: yes. On-call clears the mind and forces you to focus on what's truly important upfront ;)
<dkubb>
yeah, it's human nature. a tool never gets tired and never gets lazy
<jessekempf>
A tool is never under pressure to ship, either.
<dkubb>
in ROM code we actually fail the build if the tools flag issues
<jessekempf>
And a fast enough tool can be run from Guard, so you don't have a build-up of crufty code.
<dkubb>
I think mbj started spiking out guard-mutant
<jessekempf>
For a fast enough test suite that would be pretty spiffy.
<Gibheer>
jessekempf: so true. No developer has to do that here. If something breaks and the ops guys can't fix it, it is broken until the devs wake up
<dkubb>
jessekempf: well, there's the --rspec-unit which runs all the unit tests but I think mbj is going to be working on a new strategy that attempts to run a portion of the specs only
<dkubb>
jessekempf: so say, you saved app/models/user.rb, first mutant is going to look in spec/unit/models/user/*_spec.rb (for per-method specs), then it looks for spec/unit/models/user_spec.rb and then it runs all the unit tests (which is effectively --rspec-unit)
<dkubb>
I think the spec layout could probably be made configurable at some point too
<dkubb>
the advantage to narrowing the scope like this though is that you can get away with just running the unit tests that relate to the file you're modifying, not every spec in the whole test suite.. unless that's your preference
<jessekempf>
Gibheer: One of my team's projects is extremely high-profile within the company, so we can't roll like that.
<dkubb>
it woudn't matter if the specs take 10 minutes to run if the user tests take 10 or 15 seconds
<jessekempf>
dkubb: the particular codebase I want to run mutant against is not particularly large, but a full system test is expensive (it needs to hit the filesystem a bunch and send messages through RabbitMQ) and we'd want to generate a mutant report off the full suite of tests since the unit and integration tests only get us so far.
<dkubb>
ahh ok
<dkubb>
the thing about mutant is not that it'll find something. it'll find *a lot* guaranteed. the question is usually how to prioritize it
<dkubb>
I had one codebase that I had use heckle to get full mutation coverage and mutant found over a hundred new things in it
<jessekempf>
haha, nice
<dkubb>
and that heckled codebase was probably much more well tested than the average app
<dkubb>
or actually the best tested app I've ever worked on for $word
<dkubb>
*$work
<jessekempf>
There we go. That was the problem. Mutant's working sanely against the full DDIFLib codebase without breakenating.
<dkubb>
scratch that.. it was probably tested more extensively than the best tested app I've ever *heard* of in the ruby community. I only state that to underscore how important I think mutant is and how good it is at finding issues
<dkubb>
awesome
<jessekempf>
You don't need to sell me on mutant ;)
<dkubb>
hehe
<dkubb>
fuzz testing was probably the second best thing I ever used apart from mutant
<jessekempf>
Oh. Also. Another thing that's great about automated testers is that you can't get upset with them.
<jessekempf>
It's a machine. It's not coming up with flaws in your code because it has a grudge against you and wants to make you look dumb.
<dkubb>
that's true. it cuts past all that and it's not about ego anymore
<dkubb>
code coverage is still useful in telling you what parts of the code have 0% coverage, but beyond that I don't really trust it. it's too easy to write tests that touch a ton of code and give you false metrics
langitbiru has joined #rom-rb
<dkubb>
it's also too easy for incidental coverage to inflate things
<jessekempf>
Nice. It looks like Mutant is validating one of my personal pet peeves — the use of mutable variables.
<mbj>
dkubb: why I should feak simplcov / coveralls
<mbj>
dkubb: I know mutants metrics are suboptimal when it comes to test coverage :(
zekefast1 has joined #rom-rb
zekefast has quit [Ping timeout: 245 seconds]
langitbiru has quit [Ping timeout: 264 seconds]
mbj_ has joined #rom-rb
mbj has quit [Ping timeout: 264 seconds]
mbj has joined #rom-rb
<snusnu>
yo mbj
mbj_ has quit [Ping timeout: 276 seconds]
<mbj>
snusnu: yo
<mbj>
snusnu: need that decompose compose soon
postmodern has quit [Quit: Leaving]
mbj has quit [Read error: Connection reset by peer]
solnic has quit [Quit: Leaving...]
solnic has joined #rom-rb
snusnu has quit [Quit: Leaving.]
snusnu has joined #rom-rb
mbj has joined #rom-rb
<snusnu>
yo mbj
<snusnu>
mbj: i saw you pushed a few commits, what shall i do with them? do you still plan to add some?
<snusnu>
mbj: i fixed the last remaining spec locally before i saw that you've committed stuff
<mbj>
snusnu: I only pushed devtools and rspec api related changes
<mbj>
snusnu: Push your stuff.
<snusnu>
mbj: well, i only fixed a failing spec, so .. are you still planning to do anything, or shall i continue? ;)
<mbj>
snusnu: I'll start 18:00
<snusnu>
mbj: and merge the stuff you have in
<snusnu>
mbj: ok, then i'll just merge and add some missing specs and stuff now
<mbj>
snusnu: If you fix this before and make it realeasable you help me a lot.
<snusnu>
mbj: we'll see how far i will get
<mbj>
snusnu: thx
<snusnu>
mbj: i merged your commits, but then i amended them (as one spec had a syntax error) .. now i just noticed that history is lost, and that merge commit ends up showing me as the author .. sorry for that!