<paulproteus>
I would experience no sadness, though maybe I'd want us to write a note somewhere about that, like in some Sandstorm development documentation make a list of Supported Browsers
<zarvox>
I vaguely recall kentonv saying IE (all versions) was <1% of our pageviews
<paulproteus>
+1 do it
<thejsj>
paulproteus: I'm getting this error when running this: `** SANDSTORM SUPERVISOR: Starting up grain.
<thejsj>
nginx: [emerg] bind() to 0.0.0.0:80 failed (13: Permission denied)`
<thejsj>
I'm using my own nginx config in order to try to get websockets working...
<thejsj>
Any ideas?
<zarvox>
can't bind to port 80 unless you're root - try listening on a port number >1024? (I usually use 8000)
<thejsj>
Oh, that makes sense... haha oops
<paulproteus>
(a) The static "stack" scripts show how to fix the port number
<paulproteus>
(b) maybe we should make static+lemp support websockets proxying just to make it easier for downstream people who are modifying the stacks.
<paulproteus>
(c) thanks for being a good sport, thejsj, and we owe you a drink of your choice!
<thejsj>
I like `c`. No worries! It's interesting. I'm just about avarage with linux/devops stuff which is probably why I have so many questions
<zarvox>
static probably shouldn't, but I'm all for LEMP/uwsgi supporting websockets proxying
<zarvox>
assuming there's a reasonable default config to put there
<thejsj>
BTW, I modified the static stuff to run RethinkDB and Node.js (IO.js to be exact)
<thejsj>
Everything seems to be working except redirecting websocket traffic traffic from local.sandstrom.io:6080 to localhost:8000
<thejsj>
Let me try to keep debugging this until I can turn this into a proper question...
<paulproteus>
FWIW you're going to have a random hostname in the "Host: ..." header, even though the websockets request will get passed to port 8000
gopar has quit [Remote host closed the connection]
zeroish has quit [Ping timeout: 260 seconds]
rustyrazorblade has joined #sandstorm
pwais has joined #sandstorm
pwais has quit [Client Quit]
jadewang has quit [Remote host closed the connection]
jadewang has joined #sandstorm
jadewang has quit [Remote host closed the connection]
nowhereman has joined #sandstorm
nowhere_man has quit [Ping timeout: 264 seconds]
jadewang has joined #sandstorm
jadewang has quit [Ping timeout: 256 seconds]
<kentonv>
zarvox: Seems kind of weird. Key management is the hard part. Why would I use Google's crypto with my own key management? Might as well go the rest of the way and do the crypto.
<kentonv>
zarvox: In any case, probably a moot point for us since we want to implement fine-grained crypto eventually anyhow.
jadewang has joined #sandstorm
jadewang has quit [Ping timeout: 244 seconds]
GeorgeHahn has joined #sandstorm
<dwrensha>
every time I see the message "Binary is fine; exiting", I think "Binary is fine; exciting!"
jadewang has joined #sandstorm
jadewang has quit [Ping timeout: 246 seconds]
mrshu has quit [Ping timeout: 240 seconds]
ecloud has quit [Ping timeout: 264 seconds]
mrshu has joined #sandstorm
ecloud has joined #sandstorm
jadewang has joined #sandstorm
jadewang has quit [Ping timeout: 240 seconds]
jadewang has joined #sandstorm
NOTevil has joined #sandstorm
jadewang has quit [Ping timeout: 246 seconds]
jadewang has joined #sandstorm
jadewang has quit [Remote host closed the connection]
<paulproteus>
i,i I should be using a Sandstorm-hosted pastebin I guess
<dwrensha>
that seems to indicate that we should have "o=" for all files under /opt/sandstorm/var/
<dwrensha>
Hacker CMS!
GeorgeHahn has quit [Read error: Connection reset by peer]
<paulproteus>
Yeah, so 'socket/' should be gid=GROUP but is gid=root for me
<paulproteus>
But the files *inside* socket seem OK
<paulproteus>
brb.
jadewang has joined #sandstorm
<paulproteus>
dwrensha: OK I now believe yes the problem is root:root there and will figure out how I want to fix that.
<dwrensha>
i think i have a better understanding now too
<dwrensha>
well, at least that it's the reinstall that's crucial for making this happen
<maurer>
paulproteus: I asked about that the other day and dwrensha said he just used hackercms for that
<dwrensha>
someone should definitely make a pastebin app
<dwrensha>
but hackercms is adequate
<paulproteus>
My main problem is I'm very emotionally attached typing paste-dot-something into my browser.
<paulproteus>
So I type paste.debian.net.
<maurer>
dwrensha: one thing that seems odd about trying to formulate a pastebin app as a grain is that on the one hand, you want separate perms per doc
<maurer>
dwrensha: on the other hand, you probably don't want a full grain for every pastebin
<paulproteus>
I guess if I made paste.asheesh.org redirect to my Sandstorm shell looking at the pastebin app, and then I could click "New Paste"
<maurer>
I guess you could always just half-ass it with hash-based url
<dwrensha>
maurer: oh, grain-per-paste is what *I* want, at least
<maurer>
dwrensha: So, my worry there would be that with grain-per-paste you have a bunch of random grains cluttering up your sandstorm menu
<dwrensha>
and we're going to fix that problem, which will exist regardless
<paulproteus>
That's another thing I'm unexcited about. If I could "archive" them that'd be good probably.
<maurer>
dwrensha: Also, since apps can't unprovision themselves, you can't gc
<maurer>
I dunno. Grain-per-paste is clearly right for permissions
<maurer>
but UI and resource usage seems like it would be terrifying
<paulproteus>
For UI, "archive this grain" (better yet, "this grain auto-archives after: ( ) 1 day ( ) 1 week ( ) 1 month" as part of creating the paste
<paulproteus>
(that's just my random proposal, not necessarily something I've thought through)
<maurer>
(not to mention that since there would be a grain-create on the path to producing a paste, I can't make a command line tool that sends a file to become a paste grain)
<maurer>
paulproteus: What I think might be more useful would be to give the grain the ability to archive itself
<maurer>
paulproteus: this allows the app creator to embed arbitrary logic about archiving
<paulproteus>
maurer: Yeah, I'm +1 on that; then the chooser I mentioned could be part of the "Submit paste" UI the grain provides.
<dwrensha>
maybe the thought was that not all servers would need a dev daemon?
<dwrensha>
only development servers
<dwrensha>
but now every server needs the backend socket
<kentonv>
run-bundle can only assume the installer has created the directories that it has always created for all history
<kentonv>
because updates
<kentonv>
so usually when a new directory is introduced, run-bundle ends up responsible for creating it
<paulproteus>
I think that's only true if we permit skipping releases, and we could set some max upgrade duration. In general, yes.
<kentonv>
we definitely permit skipping releases
<paulproteus>
Debian says you can skip only one release, which allows Debian packages to simplify their upgrade logic over time, so that a package doesn't have to be responsible for being able to do all of its upgrades ever.
<paulproteus>
At the moment, yeah, what you say is definitely true.
<paulproteus>
In this case, the installer seemingly should start creating this dir, so it can be sure to chown/chmod it appropriately on installing over an existing install.
<kentonv>
I think we should avoid introducing any problems for skipping releases if possible. We obviously can't test every combination of upgrading old releases but we can at least try not to cause breakages
<paulproteus>
+1
<kentonv>
I don't know, I actually think maybe the right thing to do here is to move all directory creation into run-bundle
<paulproteus>
That's OK with me, I think.
<kentonv>
and make sure it's idempotent
<kentonv>
though if someone installs over an existing install and chooses a different server user or group, that's problematic
<paulproteus>
I was just thinking through that and what we could do, yeah.
<paulproteus>
I'd be pretty OK with "bail out and tell people to run some command".
<paulproteus>
Like "eek, this dir has permissions we are surprised by! Do this thing if you are sure this is reasonable."
<paulproteus>
permissions/ownership/etc.
<paulproteus>
I haven't yet made install.sh actively bail out if it finds an existing Sandstorm install.
<paulproteus>
I can. I feel very slightly bad about doing it. We could make it only bail out if you e.g. change SERVER_USER.
<kentonv>
we could design the code such that if the directory exists, install.sh infers the user/group that it should use based on the existing files, instead of letting the user decide.
<paulproteus>
I think dwrensha was saying that he likes to re-run install.sh sometimes over an existing install?
<paulproteus>
It reads that from the conf file, but still prompts the user, as I understand the current behavior (testing now)
<kentonv>
though permissions being screwed up is possibly a case where people will try to over-install to fix it
<paulproteus>
OK, right now, it doesn't prompt for SERVER_USER if you have an existing install, which is what we want.
<paulproteus>
Even if you ask it to prompt you.
<kentonv>
does it prompt for anything?
<paulproteus>
Er wait I'm wrong hmm.
<kentonv>
the original install script would reuse your sandstorm.conf if it existed and not prompt for any of the options again
<kentonv>
but I think that's actually bad
<paulproteus>
To reproduce the old behavior, I need to set DEFAULT_SERVER_USER to the SERVER_USER from the sandstorm.conf if there is one.
<paulproteus>
Right now it sort of lies to you )-: and says "I'm going to use $DEFAULT_SERVER_USER (which is sandstorm)" and then you are like "OK" and then it uses $SERVER_USER from the conf file.
<paulproteus>
s/sort of //
<paulproteus>
And it also doesn't let you choose a new SERVER_USER interactively.
<paulproteus>
(Verified by running code and also by reading it.)
<paulproteus>
I should fix the bug that is the lie.
<paulproteus>
Is there any reason to support re-running install.sh on top of an existing install?
<paulproteus>
If not, I could make the bug go away by telling people that they can't do that.
<kentonv>
yes: it's what users do when they want to "fix" their install
<kentonv>
or change the settings they chose at install time
<paulproteus>
Yeah, but depending on the kind of fix they want to do, we can tell them a different way to do it.
<paulproteus>
One thing people seem to want to do is sandcats-ify an install, for example, via e.g. re-install.
<kentonv>
I suppose we could detect if sandstorm is already installed and then jump into a different menu of options
<paulproteus>
I'd be happier to write Meteor code that lets people to do that, so I can write it in a more reasonable programming language, for example.
<paulproteus>
I do prefer "repair" as the verb, rather than "install over".
<paulproteus>
If I "install over", then it's unclear if my DB stays around, for example.
<paulproteus>
Honestly I expected that doing an install "over" an existing Sandstorm would remove the DB.
<kentonv>
well, most of what the installer does is configure the things that need to be configured just to get to meteor...
<paulproteus>
Fair.
<paulproteus>
But yeah, a separate menu I could totally live with.
<kentonv>
I think a lot of people expect "install over" does not damage user data
<kentonv>
especially given that e.g. apt-get never touches user data
<paulproteus>
I could totally live with a shell menu that was part of install.sh that's a separate codepath from the "install", call it "Repair/modify install", that offers e.g. "Sandcats-ify your install", and whatever else we want.
<paulproteus>
In the case that triggered this, do we know why the nice person did a reinstall?
<paulproteus>
wrong URL initially
<kentonv>
yeah
<paulproteus>
That's a pretty reasonable reason to reinstall, but they probably didn't know they could fix it by editing sandstorm.conf.
<kentonv>
I think the main things are: change BASE_URL, re-download package, and "repair" user data (e.g. try to fix up permissions issues)
<kentonv>
ideally people shouldn't need to learn about sandstorm.conf
<paulproteus>
I'm torn because we don't leave install.sh behind in people's Sandstorm dir.
<paulproteus>
If we did, and called it /opt/sandstorm/repair-or-modify-this-install.sh then I'd be more excited about making that the official way to change Sandstorm settings.
<kentonv>
I mean I'm not saying we should prevent people from editing sandstorm.conf
<paulproteus>
I can see why "Edit sandstorm.conf" is sub-awesome, since it's easy for people to not know how to use a command line editor, and easy for people to set settings that make no sense and find themselves extra stuck.
<kentonv>
but we should acknowledge that if people configured something in the installer, then they change their mind about that setting later, the thing they're likely to do is re-run the installer
<kentonv>
they're not going to look for an installer under /opt/sandstorm, they're going to re-run it the same way they did before
<paulproteus>
Part of me wants to say: "When people re-run the installer over an existing install, and choose the menu option for changing the base URL, we could tell them, 'You can do that yourself by editing /opt/sandstorm/sandstorm.conf. Go do that.'"
<paulproteus>
Good point re: the mindset of why people would re-run the install script from where they got it.
<kentonv>
I think we can very easily re-run the code that prompts for all the conf settings, and users will like that better.
<kentonv>
editing foreign config files is scary
<paulproteus>
But if there are existing settings that conflict with the new settings, I don't want to be responsible for writing+testing the code that adjusts permissions/etc. from the old situation to the new situation.
<paulproteus>
And if they change the BASE_URL it has the effect of breaking sharing links. I guess I should warn people about that if they do that?
<kentonv>
I think people are likely to intuitively understand that
<paulproteus>
I am OK being responsible for that if we think it's really important. I just want to try to convince you it's more trouble than it's worth, and telling people "You can either remove the existing install and try again, or modify this conf file" is a pretty OK thing to say.
<kentonv>
and most likely they'd mainly do this whole process fairly early on in the server's lifetime
<paulproteus>
Right, agreed, in which case they are likely OK throwing away the install too.
pwais has joined #sandstorm
<paulproteus>
ohai pwais
<kentonv>
well I'm not saying this needs to be prioritized right now
<paulproteus>
I'm at seat #25 want to hang out with me at seat #24?
* paulproteus
nods
<kentonv>
in fact I'd say don't worry about any of this right now. Just think about it.
<paulproteus>
Heh.
<paulproteus>
But for now, I do feel sad about the misleading current state, so I might add bailing-out-if-there-is-an-install.
<pwais>
Asheesh / @paulproteus are you still going to be at Workshop Cafe this afternoon? Is it at Montgomery and Bush?
<paulproteus>
Yup
<paulproteus>
Gonna be here till 6ish, possibly a smidge later
<pwais>
cool, gonna head over and I'll ping you when I get there (20-40 mins)
<pwais>
cheerios 8)
<paulproteus>
Awesome (-:
pwais has quit [Client Quit]
bb010g has joined #sandstorm
pwais has joined #sandstorm
<paulproteus>
FWIW I'm still in seat #25, might be wearing a frown as I fight with Meteor test framework stuff but happy to chat. Could move somewhere else if there's seats available near you.
<paulproteus>
pwais: ^
<pwais>
@paulproteus I'm at workshop now, sitting outside, in a yellow shirt. im reading some docs for a bit (and cooling off lol) but I'll come meet / poke you w/in 15-30 mins. or feel free to come say hi if you get bored :)
<paulproteus>
Hah, awesome (-: (long bike ride?)
<paulproteus>
I could use some outdoors myself.
NOTevil has quit [Quit: delete $self;]
treyhunner has quit [Quit: No Ping reply in 180 seconds.]
treyhunner has joined #sandstorm
isd has joined #sandstorm
isd has quit [Quit: Leaving.]
<paulproteus>
I really can't figure out how to have Meteor+Velocity actually exit.
<paulproteus>
I'm going to add a ridiculous hack and move on.
<zarvox>
paulproteus: how tied are you to Meteor here anyway? could you just use node?
<kentonv>
zarvox: meteor's build system, use of fibers, and Mongo bindings are non-trivial to replace
<maurer>
kentonv: tell me about it :P
<paulproteus>
I could but I already have data in Mongo )-:
<maurer>
(I tried to hack off the build system in order to install it on nixos)
<zarvox>
ahhh, I thought you stored data in MySQL for sandcats
<paulproteus>
why not both : P
<zarvox>
I guess you store more than just the zonefiles though
<paulproteus>
Yeah, ownership specifically.
<zarvox>
and the key-to-userid-and-recovery-mapping lives in Mongo instead
<zarvox>
Alas.
<paulproteus>
Poor pwais wants to use FUSE
<paulproteus>
or at least wants files
<paulproteus>
So he can reindex things in OpenGrok, and is sad to hear he probably would have to copy the e.g. git repos around
<paulproteus>
"Is FUSE at least on the roadmap?" he asks, and I have to say no, but something _like_ it probably is.
<kentonv>
what is the back-end for the FUSE here?
<paulproteus>
A git repo exported by some other grain, like some kind of git grain
<pwais>
*embarassed* :))
<paulproteus>
: D hi pwais
<paulproteus>
Er, the files *from* a git repo, basically
<kentonv>
hmm, why is fuse better than pulling the repo?
<paulproteus>
Fewer copies
<kentonv>
how so? FUSE still does a copy, just on-demand...
<paulproteus>
I think the idea is
<paulproteus>
If pwais had a git repo from grain1 exposed as /grain1 to his grain2, then grain2 could inotify on the /grain1 directory, and get woken up when there's new files to index.
<pwais>
FWIW my most recent use of sshfs was to mount dev code into a k8s-managed build container and have the container build / distro among the cluster. so the cluster can run pre-committed code. the idea there is that updates on the local machine get propagated to the container easily and implicitly. ive tried doing something similar using rsync and
<pwais>
it was a pain
<paulproteus>
He could instead receive a ping saying he needs to "cd /copy-of-grain1 ; git pull"
<paulproteus>
In which case, okay, but now he needs to worry about using up lots of disk space for /copy-of-grain1.
<pwais>
idk if the use case is super relevant to OpenGrok, I was mainly curious about the roadmap :)
<pwais>
OpenGrok I really wanted just to browse capnp code :)
<kentonv>
well, the thing is, FUSE is a really risky attack surface to expose.
<kentonv>
(also at the moment it's actually impossible to use FUSE in a userns container -- Linux kernel limitation -- but presumably that will be fixed someday)
<paulproteus>
Right, yeah, so I said no. (-: I hope we can have an LD_PRELOAD for fake filesystem over p9fs or who knows.
<kentonv>
LD_PRELOAD is much more plausible, yes. Or, in many cases, it probably isn't too hard to edit the app's code to make an RPC rather than read from disk.
<paulproteus>
pwais fwiw didn't realize /opt/app == his app, interestingly, which meant he wasn't sure where to look for the things his build.sh was downloading
<paulproteus>
the answer was "right under his nose!"
<zarvox>
Yeah, vault is self-hostable; it appears to export an HTTP API, so that might make a decent fit for sandstorm
<paulproteus>
"Dynamic Secrets: Vault can generate secrets on-demand for some systems, such as AWS or SQL databases. "
<paulproteus>
This is neat - like an API authorization proxy to AWS.
<paulproteus>
That's vaguely reminiscent of the Sandstorm grain sharing model.
<paulproteus>
My hack works. Hooray.
<paulproteus>
Well, still need to test on travis-ci.
<zarvox>
Hmm. When I turn the sidebar links into actual <a href> tags, only the text remains clickable. If I make the <a> be display: block, then I get a kinda-ugly dotted-purple outline around the box after clicking it.
<kentonv>
zarvox: you want to use display: block. You can clear any other weird styling the browser does by default.
<kentonv>
zarvox: the outline is probably from :active, or possibly :visited
<judytuna>
awesome. thanks, zarvox. i'm till looking through their docs and couldn't find the answer quickly =)