<cloud69>
something like `PAGES.each do |site|; scrape.scrape_post_links(site[url], site[category]); end` maybe?
<cloud69>
Would like to include category so I can do like `Category.first_or_create!(name: category, user: User.find_by_username("cloud69"))`. Updated the gist
ramfjord has quit [Ping timeout: 245 seconds]
neshpion has quit [Remote host closed the connection]
neshpion has joined #ruby
va5c0 has quit [Ping timeout: 276 seconds]
hiroaki_ has quit [Ping timeout: 272 seconds]
<cloud69>
or maybe like this, *updates the gist*
dg1727 has joined #ruby
<cloud69>
anyway thanks weaksauce and leftylink, i'm gonna try run with this
drusepth has joined #ruby
<dg1727>
I have a simple I/O question: (yes, I've done a Web search, and had trouble finding an understandable answer) I'm writing a simple commandline app. I want to start an external program (without using a shell) and ...
<dg1727>
... whatever is on my app's stdin, I want to pass directly to the external program's stdin without intervention. The external pgm's stderr can just go to the terminal window. ...
jenrzzz has quit [Ping timeout: 276 seconds]
fercell has quit [Ping timeout: 272 seconds]
<dg1727>
... My app will be processing the external pgm's stdout, which is lines of text. I want to read 1 line @ a time from the external pgm while the ext.pgm is still running, instead of having to wait for the external pgm to finish.
<dg1727>
Can you suggest what method would be recommended for this?
<dg1727>
This is on Linux
<nakilon>
dg1727 probably module Open3
<dg1727>
I will browse that on the Web right now
jenrzzz has joined #ruby
<nakilon>
its methods have the :stdin_data args for passing a text to external program stdin
<dg1727>
That sounds useful, I will look for that as well
<dg1727>
I was actually just about to ask you that question :)
jenrzzz has quit [Ping timeout: 245 seconds]
claw has quit [Read error: Connection reset by peer]
<dg1727>
nakilon: popen2 seems to be what I want. It looks like I can do my stdin redirection by specifying :in=>:in in the "opts" hash? (I can try it, of course, but I hadn't found this discussed on the Web, & if I'm mistaken about "opts" ...
<dg1727>
... you might save me a bit of trial-and-error)
claw has joined #ruby
jenrzzz has joined #ruby
<dg1727>
The input could be large, so I'd rather not buffer it into a string or the like
Swyper has joined #ruby
Swyper has quit [Remote host closed the connection]
Swyper has joined #ruby
Swyper has quit [Remote host closed the connection]
supercoven has joined #ruby
supercoven_ has quit [Ping timeout: 276 seconds]
Rudd0 has quit [Remote host closed the connection]
<nakilon>
maybe some StringIO will be fine instead of String
<nakilon>
idk, didn't try
cuerbot has joined #ruby
Swyper has joined #ruby
Swyper has quit [Remote host closed the connection]
Swyper has joined #ruby
Swyper has quit [Remote host closed the connection]
elcuervo has quit [Ping timeout: 276 seconds]
jenrzzz_ has joined #ruby
Swyper has joined #ruby
Swyper has quit [Remote host closed the connection]
jenrzzz_ has quit [Ping timeout: 264 seconds]
<weaksauce>
np cloud69
gearnode has quit [Ping timeout: 272 seconds]
bamdad has quit [Remote host closed the connection]
ramfjord has joined #ruby
bamdad has joined #ruby
ramfjord has quit [Ping timeout: 276 seconds]
Swyper has joined #ruby
Swyper has quit [Remote host closed the connection]
Swyper has joined #ruby
Swyper has quit [Remote host closed the connection]
al2o3-cr has quit [Ping timeout: 256 seconds]
al2o3-cr-tmp has joined #ruby
dpl has quit [Ping timeout: 265 seconds]
al2o3-cr-tmp has quit [Ping timeout: 256 seconds]
drusepth has quit [Ping timeout: 264 seconds]
Swyper has joined #ruby
Swyper has quit [Remote host closed the connection]
al2o3-cr-tmp has joined #ruby
Swyper has joined #ruby
Swyper has quit [Remote host closed the connection]
<cloud69>
Right now it only manages to scrape the first site, and I'm often given the warning `already initialized constant ScrapeCategory` and `previous definition of ScrapeCategory was here`
neshpion has quit [Quit: neshpion]
Swyper has joined #ruby
Swyper has quit [Remote host closed the connection]
Swyper has joined #ruby
Swyper has quit [Remote host closed the connection]
jenrzzz_ has joined #ruby
sylario has quit [Quit: Connection closed for inactivity]
jenrzzz_ has quit [Ping timeout: 264 seconds]
neshpion has joined #ruby
GodFather has quit [Ping timeout: 276 seconds]
neshpion has quit [Remote host closed the connection]
neshpion has joined #ruby
neshpion has quit [Client Quit]
neshpion has joined #ruby
jenrzzz_ has joined #ruby
Rudd0 has joined #ruby
ramfjord has joined #ruby
ChmEarl has quit [Quit: Leaving]
jenrzzz_ has quit [Ping timeout: 264 seconds]
Swyper has joined #ruby
gix has joined #ruby
ramfjord has quit [Ping timeout: 260 seconds]
gix- has quit [Ping timeout: 265 seconds]
drusepth has joined #ruby
Swyper has quit [Ping timeout: 240 seconds]
adu_ has joined #ruby
supercoven has quit [Read error: Connection reset by peer]
supercoven has joined #ruby
supercoven has quit [Max SendQ exceeded]
supercoven has joined #ruby
supercoven has quit [Max SendQ exceeded]
supercoven has joined #ruby
supercoven has quit [Max SendQ exceeded]
supercoven has joined #ruby
supercoven has quit [Max SendQ exceeded]
jenrzzz_ has joined #ruby
adu_ has quit [Quit: adu_]
jenrzzz_ has quit [Ping timeout: 260 seconds]
orbyt_ has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
m27frogy has quit [Ping timeout: 264 seconds]
orbyt_ has joined #ruby
acsims has quit [Ping timeout: 264 seconds]
actually1atman has joined #ruby
acsims has joined #ruby
acsims has quit [Ping timeout: 276 seconds]
orbyt_ has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
jenrzzz has quit [Ping timeout: 260 seconds]
jenrzzz has joined #ruby
jenrzzz has quit [Ping timeout: 245 seconds]
Xiti has quit [Quit: Leaving]
Swyper has joined #ruby
ramfjord has joined #ruby
jenrzzz has joined #ruby
Swyper has quit [Remote host closed the connection]
adu_ has joined #ruby
Swyper has joined #ruby
neshpion has quit [Quit: neshpion]
Swyper has quit [Remote host closed the connection]
ruurd has joined #ruby
dg1727 has quit [Remote host closed the connection]
dg1727 has joined #ruby
Swyper has joined #ruby
Swyper has quit [Ping timeout: 264 seconds]
jenrzzz has quit [Ping timeout: 240 seconds]
jenrzzz has joined #ruby
jenrzzz has quit [Ping timeout: 276 seconds]
adu_ has quit [Quit: adu_]
jenrzzz has joined #ruby
cd has quit [Quit: cd]
jenrzzz has quit [Ping timeout: 260 seconds]
jenrzzz has joined #ruby
acsims has joined #ruby
jenrzzz has quit [Ping timeout: 264 seconds]
jenrzzz has joined #ruby
postmodern has joined #ruby
jenrzzz has quit [Ping timeout: 265 seconds]
jenrzzz has joined #ruby
Xiti has joined #ruby
cloud69 has quit [Quit: Connection closed for inactivity]
shtirlic has joined #ruby
acsims has quit [Ping timeout: 260 seconds]
acsims has joined #ruby
jenrzzz has quit [Ping timeout: 260 seconds]
jenrzzz has joined #ruby
coniptor has quit [Ping timeout: 256 seconds]
coniptor has joined #ruby
acsims has quit [Ping timeout: 276 seconds]
acsims has joined #ruby
acsims has quit [Ping timeout: 240 seconds]
drusepth has quit [Ping timeout: 276 seconds]
jenrzzz has quit [Ping timeout: 245 seconds]
xco has joined #ruby
jenrzzz has joined #ruby
al2o3-cr-tmp has quit [Ping timeout: 256 seconds]
acsims has joined #ruby
s3nd1v0g1us has quit [Quit: WeeChat 3.0.1]
akem_ has joined #ruby
jenrzzz has quit [Ping timeout: 264 seconds]
jenrzzz has joined #ruby
zolferino has joined #ruby
zolferino has quit [Excess Flood]
jenrzzz has quit [Ping timeout: 245 seconds]
xco has quit [Quit: My Mac Mini has gone to sleep. ZZZzzz…]
jenrzzz has joined #ruby
zolferino has joined #ruby
fercell has joined #ruby
jenrzzz has quit [Ping timeout: 276 seconds]
zolferino has joined #ruby
zolferino has quit [Excess Flood]
jenrzzz has joined #ruby
xco has joined #ruby
podge has joined #ruby
andremedeiros has quit [Read error: Connection reset by peer]
andremedeiros has joined #ruby
zolferino has joined #ruby
zolferino has quit [Excess Flood]
xco has quit [Quit: My Mac Mini has gone to sleep. ZZZzzz…]
xco has joined #ruby
jenrzzz has quit [Ping timeout: 240 seconds]
jenrzzz has joined #ruby
podge has quit [Ping timeout: 246 seconds]
podge has joined #ruby
akem has quit [Quit: leaving]
akem_ has quit [Quit: Leaving]
akem has joined #ruby
akem has quit [Client Quit]
akem has joined #ruby
Swyper has joined #ruby
akem has quit [Client Quit]
akem has joined #ruby
Swyper has quit [Ping timeout: 245 seconds]
shtirlic has quit [Ping timeout: 260 seconds]
jenrzzz has quit [Remote host closed the connection]
shtirlic has joined #ruby
fercell has quit [Read error: Connection reset by peer]
jenrzzz has joined #ruby
fercell has joined #ruby
al2o3-cr-tmp has joined #ruby
fercell has quit [Ping timeout: 264 seconds]
hiroaki_ has joined #ruby
dionysus69 has joined #ruby
dionysus69 has joined #ruby
jenrzzz has quit [Ping timeout: 240 seconds]
al2o3-cr-tmp has quit [Client Quit]
al2o3-cr has joined #ruby
xco has quit [Quit: My Mac Mini has gone to sleep. ZZZzzz…]
gray-_-wolf has quit [Quit: WeeChat 3.0]
jenrzzz has joined #ruby
Rounin has joined #ruby
jenrzzz has quit [Ping timeout: 245 seconds]
ramfjord has quit [Ping timeout: 240 seconds]
gearnode has joined #ruby
ramfjord has joined #ruby
Swyper has joined #ruby
m27frogy has joined #ruby
ramfjord has quit [Ping timeout: 276 seconds]
Swyper has quit [Remote host closed the connection]
Swyper has joined #ruby
Swyper has quit [Ping timeout: 260 seconds]
roadt__ has quit [Ping timeout: 246 seconds]
roadt__ has joined #ruby
Swyper has joined #ruby
postmodern has quit [Quit: Leaving]
ramfjord has joined #ruby
ramfjord has quit [Ping timeout: 260 seconds]
_whitelogger has joined #ruby
jenrzzz has quit [Ping timeout: 245 seconds]
GankMove has joined #ruby
coniptor has quit [Read error: Connection reset by peer]
jenrzzz has joined #ruby
jenrzzz has quit [Ping timeout: 245 seconds]
coniptor has joined #ruby
dionysus69 has quit [Ping timeout: 240 seconds]
al2o3-cr has quit [Read error: Connection reset by peer]
podge has quit [Quit: quit]
al2o3-cr has joined #ruby
coniptor has quit [Read error: Connection reset by peer]
cuerbot has quit [Read error: Connection reset by peer]
elcuervo has joined #ruby
coniptor has joined #ruby
hiroaki_ has quit [Ping timeout: 246 seconds]
yann-kaelig has joined #ruby
yann-kaelig has quit [Client Quit]
DaRock has joined #ruby
fercell has joined #ruby
akem has quit [Ping timeout: 245 seconds]
salvor has quit [*.net *.split]
isene has quit [*.net *.split]
roger_rabbit has quit [*.net *.split]
salvor has joined #ruby
isene has joined #ruby
roger_rabbit has joined #ruby
ramfjord has joined #ruby
ramfjord has quit [Ping timeout: 260 seconds]
cd has joined #ruby
<phaul>
&renick
rubydoc_ is now known as rubydoc
<rubydoc>
okay
jenrzzz has joined #ruby
jeromelanteri has joined #ruby
jenrzzz has quit [Ping timeout: 240 seconds]
shtirlic has quit [Ping timeout: 264 seconds]
AndreYuhai has joined #ruby
ramfjord has joined #ruby
<AndreYuhai>
Hey there, can one Sidekiq process handle different jobs in the same queue?
<AndreYuhai>
For example one job is to send an email to a user and the other is to fetch some data from the web.
<AndreYuhai>
I've always been creating separate sidekiq processes for each job type and now I am wondering whether I've been wasting resources the whole time. :D
andremedeiros has quit [Read error: Connection reset by peer]
andremedeiros has joined #ruby
GodFather has joined #ruby
<c355e3b>
you don't need to split sidekiq processes per job type
ramfjord has joined #ruby
rwb1 has joined #ruby
sombra has quit [Remote host closed the connection]
rwb has quit [Ping timeout: 264 seconds]
rwb1 is now known as rwb
ramfjord has quit [Ping timeout: 240 seconds]
<adam12>
AndreYuhai: I wouldn't run separate sidekiq processes unless I wanted one for a specific queue only (ie. something extremely long running). Or maybe if I had a customer that was notorious for creating huge import jobs that would consume the tie up the queue for everyone else.
<havenwood>
nakilon: Probably not the type of meaningful improvement you're looking for, but you save a tiny bit with: t = Array.new all.size do |i|
<havenwood>
nakilon: all.dig(i, g)
<havenwood>
nakilon: Depending on implementation, might gain a smidge from `.abs2` instead of ` ** 2`.
<havenwood>
nakilon: Which is the slow part? Is `set` or `all` of a known size?
<havenwood>
I guess `all` isn't, since you check its size.
sombra has joined #ruby
<havenwood>
Size range*
<havenwood>
I mean, are `set` or `all` within known ranges of size. Just curious.
hiroaki_ has joined #ruby
aloy has quit [Ping timeout: 240 seconds]
ramfjord has joined #ruby
adu_ has joined #ruby
dionysus69 has quit [Ping timeout: 260 seconds]
ramfjord has quit [Ping timeout: 264 seconds]
GodFather has quit [Ping timeout: 240 seconds]
<AndreYuhai>
Alright thank you c355e3b, adam12. I didn't know one process can handle any type of job. That's probably because I was always running sidekiq using -r and some script.
GodFather has joined #ruby
DaRock has quit [Ping timeout: 240 seconds]
Swyper has quit [Remote host closed the connection]
jenrzzz has joined #ruby
jenrzzz has quit [Ping timeout: 245 seconds]
Swyper has joined #ruby
Swyper has quit [Remote host closed the connection]
Swyper has joined #ruby
Swyper has quit [Remote host closed the connection]
Xiti` has quit [Quit: Leaving]
Swyper has joined #ruby
ramfjord has joined #ruby
romanlevin333236 has quit [Ping timeout: 240 seconds]
<AndreYuhai>
weaksauce, checking that now, thanks!
<nakilon>
is basically a subset of possible indexes to take from "all"
dinfuehr has quit [Ping timeout: 265 seconds]
<weaksauce>
AndreYuhai no problem... as an aside that template is a pretty good start for a decently configured app with an admin area and some bells and whistles included
dinfuehr has joined #ruby
<AndreYuhai>
weaksauce, I don't know much about Rails but I really like Chris' videos. :D I will be using jumpstart as well.
<AndreYuhai>
Thanks for the suggestion
<weaksauce>
no problem
<weaksauce>
the only downside to his template is it does require some extra servers out of the box like postgres and redis but that's not too much to add
jenrzzz has joined #ruby
jenrzzz has quit [Ping timeout: 240 seconds]
<nakilon>
that (a-b)**2 line takes 40% of the run time
raul782 has joined #ruby
hiroaki_ has quit [Ping timeout: 272 seconds]
<nakilon>
I guess I'll precalculate a (arr.size x arr.size) array of (a-b)**2
raul782 has quit []
<nakilon>
it's now 65%, lol
fercell has quit [Ping timeout: 264 seconds]
Xiti has joined #ruby
hiroaki_ has joined #ruby
drusepth has joined #ruby
ByronJohnson has quit [Ping timeout: 272 seconds]
kristian_on_linu has quit [Remote host closed the connection]
ByronJohnson has joined #ruby
yann-kaelig has joined #ruby
yann-kaelig has quit [Client Quit]
jla has joined #ruby
jla has quit [Ping timeout: 240 seconds]
adu_ has quit [Quit: adu_]
Swyper has quit [Remote host closed the connection]
ramfjord has joined #ruby
gray-_-wolf has joined #ruby
TCZ has joined #ruby
GodFather_ has joined #ruby
GodFather_ has quit [Client Quit]
GodFather_ has joined #ruby
GodFather_ has quit [Remote host closed the connection]
GodFather_ has joined #ruby
GodFather_ has quit [Read error: Connection reset by peer]
ramfjord has quit [Ping timeout: 240 seconds]
GodFather has quit [Quit: Ex-Chat]
GodFather has joined #ruby
GodFather has quit [Client Quit]
GodFather has joined #ruby
Swyper has joined #ruby
gray-_-wolf has quit [Quit: WeeChat 3.0]
cow[moo] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
ramfjord has joined #ruby
s3nd1v0g1us has joined #ruby
ramfjord has quit [Ping timeout: 260 seconds]
work_ has joined #ruby
drusepth has quit [Ping timeout: 264 seconds]
drusepth has joined #ruby
cow[moo] has joined #ruby
jenrzzz has joined #ruby
Swyper has quit [Remote host closed the connection]
s3nd1v0g1us has quit [Ping timeout: 245 seconds]
GodFather has quit [Remote host closed the connection]
GodFather has joined #ruby
s3nd1v0g1us has joined #ruby
jenrzzz has quit [Ping timeout: 264 seconds]
jenrzzz has joined #ruby
jenrzzz has quit [Ping timeout: 260 seconds]
Xiti has quit [Quit: Leaving]
_whitelogger has joined #ruby
kristian_on_linu has joined #ruby
Xiti has joined #ruby
Rudd0 has joined #ruby
nofxx_ has joined #ruby
jenrzzz has joined #ruby
nofxx has quit [Ping timeout: 276 seconds]
cow[moo] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
jenrzzz has quit [Ping timeout: 240 seconds]
cloud69 has joined #ruby
GankMove has joined #ruby
drusepth has quit [Ping timeout: 240 seconds]
GankMove has quit [Max SendQ exceeded]
GankMove has joined #ruby
GankMove has quit [Max SendQ exceeded]
TCZ has quit [Quit: WRFMP MRFVR MBONP MLZLN]
neshpion has joined #ruby
ramfjord has joined #ruby
ramfjord has quit [Ping timeout: 245 seconds]
jenrzzz has joined #ruby
gearnode has joined #ruby
jenrzzz has quit [Ping timeout: 245 seconds]
Guest47370 has joined #ruby
Guest47370 has left #ruby ["WeeChat 1.9.1"]
drusepth has joined #ruby
AndreYuhai has quit [Quit: Leaving]
work_ has quit [Quit: Connection closed for inactivity]
hiroaki_ has quit [Ping timeout: 272 seconds]
shtirlic has quit [Read error: Connection reset by peer]