<qrush>
i'm worried not notifying authors here might cause a backlash
TheComrade has quit [Quit: TheComrade]
<drbrain>
yeah, we should open a help.rubygems.org ticket and pull them in
<qrush>
(i've already deleted a few)
vertis has joined #rubygems
<qrush>
one side of me knows these folks did this for a quick laugh and thats it, and they wont miss it
<vertis>
qrush: sorry, only just saw your message
<qrush>
vertis: hey, no worries
<vertis>
what's up?
<qrush>
needed console access, we're good. although i'm not sure if drbrain got wired up
<drbrain>
not yet, samkottler was going to hook me up tomorrow
<vertis>
Added to the list of SSH keys on the server?
<drbrain>
I made a pull request to rubygems-aws but it didn't get deployed
<drbrain>
17:01 samkottler: drbrain: I'll roll out your user tomorrow morning since I have 0% battery left
<vertis>
hrummm, looks
<samkottler>
vertis: go for it
* samkottler
is on his phone now
<qrush>
drbrain: ok, well i've deleted a few but there's still several on that list
<vertis>
I'll see what I can do. I'm on a new laptop
<drbrain>
vertis: no big
<qrush>
we could deploy a "Report Abuse" link that creates a new discussion on tender
<qrush>
and report them..but then what? give them a chance to delete it themselves?
<drbrain>
I haven't needed ssh/console access yet
<drbrain>
qrush: same as we do for renames
<qrush>
we should have some kind of policy here
<drbrain>
have a discussion, check evidence, make a decision based on what a reasonable person would think
<drbrain>
we did it for posterous and, while slow, it worked out fine
<samkottler>
how about we write the blog post and then talk to people
<samkottler>
they'll have context then
<samkottler>
I'm happy to draft it
<qrush>
1. Report Abuse, 2. Check based on gem name, ask gem author to manually remove, 3. No response in a week? Permadelete
<qrush>
4. Lock gem namespace?
<vertis>
drbrain, I'll do the deployments so you can ssh
<vertis>
give me 20 mins
<drbrain>
I think: 1. Report Abuse, 2. Invite author to help.rubygems.org, 3. Check based on gem name, README, tests 3. Decide to permadelete or not based on what a reasonable person would decide after a week
<drbrain>
it's what judges do
<qrush>
i hate that this is "policing" the gem namespace
<drbrain>
yes
<qrush>
but we're very past the "so we're nice" phase here
<drbrain>
and we should need at least a simple majority to decide on permadelete/lock
<qrush>
simple majority of whom?
<drbrain>
you, me, evan, other people we invite as stewards of the namespace
<qrush>
blog post should mention the google group, to discuss plans there
<qrush>
or at least "hey we're thinking of doing this"
<drbrain>
I think we should have around 5 stewards
<drbrain>
yeah
<drbrain>
transparency++
<qrush>
i have a strong feeling the "hoe" name will be challenged if we do this
<drbrain>
plus I've been making tons of bugfix releases lately
<drbrain>
I'm up to 22 since 2.0, I think
<qrush>
TOP SCORE
<vertis>
qrush, drbrain: just deploying the LB now, will get you to test in a second
<drbrain>
vertis: I have to catch a bus in 10 minutes (but don't rush on my behalf)
<qrush>
ok, trying to get the app working locally so i can run this task.
<vertis>
drbrain pressure pressure
<vertis>
drbrain: back in the chatroom after, or is that it for the day
<drbrain>
vertis: I should be back in an hour, maybe two
<vertis>
okay
<vertis>
Run complete
<vertis>
drbrain: try logging into 54.245.255.174
<vertis>
with your username
<qrush>
Looks Good On My Machine™
<drbrain>
drbrain@balancer02:~$
<vertis>
cool
<vertis>
I'll do the other boxes
TheComrade has joined #rubygems
<qrush>
also, wow, ruby is way out of date :(
cowboyd has joined #rubygems
<vertis>
-_-
<vertis>
we really need a deploy box
<drbrain>
with last year's rubygems and rdoc work during the push for ruby 2.0.0 falling of my github user page, it's getting difficult to reach the 2500 commits
<qrush>
agreed, we are still missing staging
cmeiklejohn has joined #rubygems
<qrush>
we hit the iceberg but we're sailing with half a ship still
<vertis>
well it won't be this weekend
<vertis>
railsrumble
<qrush>
Eh?
<vertis>
but I should touch base on what needs doing
<dwradcliffe>
we're still missing proper backups too
<qrush>
maybe we need a "top 3" list for everything
<qrush>
this works well at 37 :)
<qrush>
there's always 1000 things to do but it's easy to focus on a small list
<drbrain>
bus time! later folks
<vertis>
I can dedicate a bunch of time to this
<qrush>
top 3 ops, dev, community concerns
<vertis>
but I do need priorities
tjbiddle has quit [Quit: tjbiddle]
<vertis>
sounds like we have some
<dwradcliffe>
sounds like a good approach
<dwradcliffe>
qrush: do you use a tool for that or just informal?
<qrush>
dwradcliffe: informal
<vertis>
app server is deployed
<vertis>
just doing dbmaster, and then I'm done
<qrush>
sweet
<vertis>
hmmmm
<qrush>
ops wise my worries are: 1) security 2) backups 3) killing chef deploys
<qrush>
Staging too :)
<vertis>
chef failed on dbmaster
<vertis>
* Error: pid file is invalid, please manually kill the stale server process
<vertis>
try running again -_-
<vertis>
rush, I'll start working on them
<vertis>
qrush*
<dwradcliffe>
vertis: I've seen that error in vagrant before
<vertis>
hopefully intermittent
DanKnox is now known as DanKnox_away
cowboyd has quit [Remote host closed the connection]
<dwradcliffe>
qrush: what do you mean by security?
<vertis>
hrrumph, that error message means I need to manually kill postgres.
<dwradcliffe>
vertis: do you really need to restart postgres though?
<vertis>
chef tries to
<vertis>
and won't complete otherwise
<vertis>
so bah
<vertis>
dwradcliffe: otherwise I would say no
<dwradcliffe>
vertis: did it complete the users part?
<vertis>
it did
<dwradcliffe>
should be good for now then, unless I'm missing something
<vertis>
but that leaves it in a fairly inconsistent state
<dwradcliffe>
true
<vertis>
also leaves it for next time
<dwradcliffe>
did it change something in postgres that triggered the restart?
<vertis>
no I don't think so
<vertis>
qrush, thoughts?
<dwradcliffe>
maybe fix the pid file manually and try chef again?
<vertis>
I looked at that
<vertis>
it seems to have the correct pid
<vertis>
dwradcliffe: lol, found your gist while googling
<jkline>
I am happy to comment on the pull request itself if you prefer.
<jkline>
hi mpapis :)
<mpapis>
jnimety, Internet problems, happens from time to time
<mpapis>
:D
<mpapis>
jkline, it's marked 2.2 - so at least weeks I think
<jkline>
mpapis: ok, i was not sure what the 2.2 desgination meant.
<jkline>
you had recommended also merging down to 2.1 branch, do you think that will occur?
<jkline>
or, is it too big a functional change for a patch revision?
<mpapis>
that's question for maintainers ...
<jkline>
yes, indeed.
<jkline>
i'll comment on the pull request and go back to patient waiting. In the mean time I did rvm rubygems head, we'll see how that works
<mpapis>
:D
<jkline>
does not work… oh well
<jnimety>
mpapis, it's so hard to know when people are joking on the internets
<mpapis>
I'm rarely, usually when I blame rvm
<jkline>
jnimety: yes, text looses something in that way. Is there some particular joke you are refering to?
<jnimety>
mpapis blaming internet problems for gem install issues. couldn't/can't tell if joking or serious
<mpapis>
jnimety, that's half serious - this happens, I call it internet problems, usually means servers are overloaded
<mpapis>
evan, ^ is that the case?
<jnimety>
anyway, over the past week or two we've been experiencing gem install issues (intermittent ssl errors and timeouts) both locally and on travis CI. If there's anything we can do do help please let me know, I'll lurk for a while
TheComrade has joined #rubygems
havenwood has quit [Remote host closed the connection]
<mpapis>
maybe imperator - do you know about the 'internet problems" ^?
tenderlove has joined #rubygems
snooc has quit []
<Defiler>
As far as I'm aware, that is an unsolved problem
<Defiler>
there are a million stack overflow posts about it, the victim is always blamed
<Defiler>
My guess is it will turn out to be a problem with openssl / net/http etc somewhere
<Defiler>
Too many people have seen it
<Defiler>
Even if it is server overload (I'd be a little surprised, given that we're talking about Amazon S3 here), it at least needs to be caught somewhere else and turned into a sane error that gets retried
<drbrain>
jkline: I've merged it to the 2.x branch, bug me Thursday if I haven't released it and I'll make a new release just for you
antichatter has joined #rubygems
<jnimety>
drbrain: I'll ask them about it but I get intermittent errors locally as well (OS X)
<drbrain>
jnimety: since timeouts come from the OS, there's little RubyGems can do about it
<drbrain>
RubyGems uses the default net/http timeouts
<imperator>
not seeing any timeouts here
<drbrain>
(I'm unsure if OpenSSL has configurable timeouts, but if it does, we use the defaults there too)
<jnimety>
drbrain: why timeouts at all?
almostwhitehat has quit []
<drbrain>
jnimety: to help users find network problems
<drbrain>
also, the OS has its own timeouts for things like connect(2)
<drbrain>
if connect(2) takes too long the OS will stop trying and give back Errno::ETIMEDOUT
<drbrain>
rubygems doesn't set any socket options to change this timeout either
<jnimety>
I'm suggesting that the root cause is not a network issue but something upstream
<jnimety>
these timeouts should not happen
jkline has quit [Quit: jkline]
<jnimety>
Defiler pointed out quite well that these issue are common. if it's a network issue why do websites not timeout as well? or other network based services
vertis has quit [Quit: vertis]
<drbrain>
I agree they should not happen, but there are probably a dozen computers between you and rubygems.org/S3
<Defiler>
rubygems needs to retry these
<drbrain>
browsers may have different retry semantics that rubygems doesn't implement
<Defiler>
at least until they are locked down to the proper exception hierarchy
<Defiler>
they get raised as things that don't look like timeouts so rubygems just bails
<drbrain>
note that ruby 2.0 automatically retries GET requests
<jnimety>
very good. I'll dig into it in the next few days.
<drbrain>
Gem::Request and Net::HTTP::Persistent have similar ancestors so it shouldn't be an excessive amount of work there
Sc0rp10n has joined #rubygems
<jnimety>
drbrain, Defiler, imperator thanks for your time
jnimety has left #rubygems ["Leaving..."]
<drbrain>
jnimety: I would like to get this fixed, but I have difficulty reproducing it in consistently
<drbrain>
aw, gone
<drbrain>
was going to say "sorry if I came off wrong in our discussion"
<imperator>
nah, you're good, he was getting worked up, clearly frustrated
<shaiguitar>
Defiler: woah, irc
<shaiguitar>
Defiler: do you have access to rubygems' db? I was talking to qrush the other day about getting a dump so I can investigate some issues outlined in https://github.com/rubygems/rubygems.org/pull/606
<qrush>
shaiguitar: hey, samkottler told me he could help you out today
<qrush>
he was feeling sick though :)
<shaiguitar>
oh hey
Elhu has quit [Quit: Computer has gone to sleep.]
<shaiguitar>
ah ok I just wasn't sure what the status was
<shaiguitar>
thx!
<shaiguitar>
samkottler: around?
<drbrain>
qrush: can you add verity to our rubygems gittip group?
<drbrain>
qrush: AFAIK you have access to the rubygems twitter password
huoxito has quit [Remote host closed the connection]
<qrush>
i actually dont have that pw
<qrush>
or the rubygems_status pw right now
huoxito has joined #rubygems
<drbrain>
ok, we'll have to wait for samkottler then
rafaelfranca has joined #rubygems
huoxito has quit [Ping timeout: 246 seconds]
jkline has quit [Quit: jkline]
cowboyd has quit [Remote host closed the connection]
vertis has joined #rubygems
ZachBeta has quit [Quit: Computer has gone to sleep.]
kanobe has quit [Read error: Operation timed out]
marpaia has quit [Quit: WeeChat 0.4.1]
marpaia has joined #rubygems
kanobe has joined #rubygems
arthurnn has quit [Remote host closed the connection]
einarj has joined #rubygems
arthurnn has joined #rubygems
jkline has joined #rubygems
arthurnn has quit [Ping timeout: 256 seconds]
einarj has quit [Ping timeout: 256 seconds]
TheComrade has joined #rubygems
jkline has quit [Quit: jkline]
snuffeluffegus has joined #rubygems
ZachBeta has joined #rubygems
redmenace has quit [Ping timeout: 252 seconds]
ZachBeta has quit [Quit: Computer has gone to sleep.]
luislavena has joined #rubygems
luislavena has quit [Remote host closed the connection]
luislavena has joined #rubygems
einarj has joined #rubygems
einarj has quit [Remote host closed the connection]
einarj has joined #rubygems
<drbrain>
mpapis: I don't know what to do about #677, so far it doesn't seem like implementers want improvements
<drbrain>
ruby implementers
<drbrain>
also, I don't really want to have regular users running `gem install --default` as there's no way to uninstall such a gem
<drbrain>
(and I don't want to implement one, either)
<Defiler>
shaiguitar: I don't have access to the production install, just the code, yeah
redmenace has joined #rubygems
einarj has quit [Remote host closed the connection]
dvu_ has quit [Remote host closed the connection]
tenderlove has quit [Remote host closed the connection]