craigp has quit [Remote host closed the connection]
craigp has joined #rubinius
<[spoiler]>
Morning
pchalupa has joined #rubinius
havenwood has quit []
|jemc| has quit [Ping timeout: 272 seconds]
djellemah has joined #rubinius
djellemah_ has joined #rubinius
djellemah_ has left #rubinius [#rubinius]
djellemah has left #rubinius [#rubinius]
flavio has joined #rubinius
flavio has joined #rubinius
dimday has quit [Quit: Leaving.]
Thijsc has joined #rubinius
elia has joined #rubinius
nirvdrum has joined #rubinius
nirvdrum has quit [Ping timeout: 252 seconds]
nirvdrum has joined #rubinius
nirvdrum_ has joined #rubinius
nirvdrum has quit [Ping timeout: 244 seconds]
<yorickpeterse>
morning
goyox86 has joined #rubinius
goyox86 has quit [Remote host closed the connection]
Bwild has quit [Remote host closed the connection]
JohnBat26 has joined #rubinius
Thijsc has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Thijsc has joined #rubinius
craigp has quit [Ping timeout: 252 seconds]
kagaro has joined #rubinius
<yopp>
morning
djinni has quit [Ping timeout: 252 seconds]
djinni has joined #rubinius
craigp has joined #rubinius
<yopp>
cremes, stupid question about mmap: let's say, I have 5 processes, and they are mmap one file. 4 can only read, and 1 can read and write to that file. If writer modifies section of the file mapped to memory, this change will be reflected to readers almost immedeately?
<yopp>
another thing, that I'm not getting: if I'm randomly reading file (lets say, reading index), I need to do read using 1 mmap'ed memory region, or I should map to one buffer for each read, or I should use map each time?
max96at|off is now known as max96at
<yopp>
or it's more like: I can mmap a file bigger than amount of memory available, and access to mmaped buffer will take care of everything? I will be able to access any part of the file, and it will copy it to memory on demand?
<DireFog>
mmap makes the file part of virtual memory space, physical memory doesn't play into it too much
jeregrine has quit []
<DireFog>
visibility of changes depends on the parameters to the mmap call
jeregrine has joined #rubinius
<yopp>
DireFog, so proper way is to map whole file, and just seek/read over it, and all the memory access will be managed by OS?
<DireFog>
the idea is to map the file and then you access it just like memory
<yopp>
so I don't need to think about how much physical memory is needed?
<DireFog>
I don't know how any OS handles paging for that
<DireFog>
it should only use a couple of pages at any given time, it doesn't read the entire file into physical memory
<yopp>
from mmap man page, seems like if you access to region of file, that is not in physical memory it will generate a page fault, and page will be loaded to memory and then it will be accessible to process
goyox86 has joined #rubinius
<yopp>
okay, that's makes sense
<yopp>
and if I'm using MAP_SHARED, then all processes will see changes immediately
<yopp>
so basically I can avoid reader blocking, if changes are atomical
<cremes>
yopp: i see that you read the man page. that should answer all of your questions.
<yopp>
I read a lot of stuff lately, and it's made a mess in my head :B
<yopp>
I'm trying to verify my assumption, since man pages can't talk
<DireFog>
goota love workdays that end with closing 5-10 RFCs
<yopp>
and seems like memory operations are not atomical, which is sad :|
<cremes>
yopp: if you need atomic operations with mmap i think you need to look into shared mutexes or semaphores. that’s outside my area of expertise.
<yopp>
that means locking :|
<cremes>
of course. how else would you do it?
<yopp>
with magic! :B
<cremes>
ha! well, you just need something to serialize the operations. a mutex is a reasonable way to accomplish that. maybe you can use pipes.
<yopp>
I'm curious how LMDB achieved this "Reader/writer transactions: readers don't block writers and writers don't block readers. Writers are fully serialized, so writes are always deadlock-free."
<cremes>
some kind of FIFO probably
<cremes>
it also probably allows “dirty reads”
<yopp>
dirty reads?
<cremes>
yes, where a read can occur in the middle of a write so you aren’t guaranteed to see the “latest” version of data. google for dirty read and it will probably give you a lot more detail.
* DireFog
recommends safe search when googling for that
<yopp>
:B
craigp has quit [Ping timeout: 245 seconds]
<yopp>
uh.
<[spoiler]>
Lmao
<yopp>
huh. Now I'm getting why our mongo it locked 75% of time.
<cpuguy83>
Lots of writes == really bad for mongo
nirvdrum_ is now known as nirvdrum
<yopp>
yep
<yopp>
okay, seems like LMDB uses "multi-version concurrency control"
<yopp>
so instead of writing to the same page, they are using new one
craigp has joined #rubinius
<cpuguy83>
2.8/3.0 (mongo) looks like it actually implements document-level locking, which is a HUGE improvement.
<yopp>
really?
<cpuguy83>
Based on the docs, it seems that way.
havenwood has joined #rubinius
nirvdrum has quit [Quit: Leaving]
<yorickpeterse>
Bingo, got an AWS multi-threading problem in MRI
<yorickpeterse>
"hurrdurr these problems don't occur on MRI therefor aws-sdk is thread-safe"
<yorickpeterse>
my butt is thread-safe
<yorickpeterse>
oh wait, this is actually JRuby even
<yopp>
do you have mutex on it? :B
<yorickpeterse>
Not in this case
<yorickpeterse>
I have on all my own apps
craigp has quit []
mustmodify has joined #rubinius
<kagaro>
brixen i fils a ticket for rubysl-date, %Z is not working as expected, its just showing %z instead
<pchalupa>
hi does initialization of an object provide any visibility guaranties, e.g. similar to final field in java
<yorickpeterse>
visibility as in?
<pchalupa>
yorickpeterse if there is a final field in java object it's guaranteed that all threads will see correct value (supplied in initializer) no synchronization needed
goyox86 has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<yorickpeterse>
Ruby has nothing like that
<yorickpeterse>
If you want synchronization you'll have to do it yourself in most cases
<mustmodify>
we like threads, but not enough. :P
<pchalupa>
MRI's GIL provides that afaik guarantee. This is an issue/question only on rubinius and JRuby
<yorickpeterse>
The GIL isn't bulletproof, you can still have race conditions on MRI
<pchalupa>
yeah sure
<mustmodify>
pchalupa: can you provide documentation on that?
<pchalupa>
but in this case you cannot se old value in @ivar afaik
<pchalupa>
mustmodify hi, on what?
<pchalupa>
java final field or MRI GIL ?
<mustmodify>
that MRI guarantees synchronization.
goyox86 has joined #rubinius
tenderlove has joined #rubinius
* pchalupa
still looking
nirvdrum has joined #rubinius
tenderlove has quit [Quit: Leaving...]
<yorickpeterse>
pchalupa: what do you mean with "but in this case you cannot se old value in @ivar afaik"?
tenderlove has joined #rubinius
<pchalupa>
e.g.: in thread1: set @a = 1 then @a = 2 then context is switched to thread2 @a will always be 2 never 1 (in MRI Ruby code)
<pchalupa>
yorickpeterse ^
<pchalupa>
sorry for poor explanation
<yorickpeterse>
That is not guaranteed I believe
<yorickpeterse>
MRI can decide to switch threads between the assignments
<yorickpeterse>
Although much of that is "undefined behaviour" as far as I know
<yorickpeterse>
in general you should not rely on any variable assignment being synchronized
<yorickpeterse>
I think constant assignments are synchornized, but I'm not 100% sure
<yorickpeterse>
but even there, if you want to share the data:
<yorickpeterse>
Mutex it
<pchalupa>
yorickpeterse yep, I know that the context switch may happen at any time, this was an example: if it happens at that place, @a will be seen as 2 in thread2.
* pchalupa
trying to look it up where I've got that.
* pchalupa
may be wrong though
[spoiler] has quit [Ping timeout: 272 seconds]
goyox86 has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<GitHub11>
rubinius/1.8.7 52442ef Brian Shirai: Merge remote-tracking branch 'origin' into 1.8.7
GitHub11 has left #rubinius [#rubinius]
<brixen>
additionally, is it specified anywhere?
<brixen>
MRI makes a guarantee about when it will or will not change GIL semantics?
<brixen>
if they do, I've never seen it
<yorickpeterse>
People claim it can switch when doing IO, and that apparently it's "sort of good" at it
<yorickpeterse>
But I've yet to see any docs on that
<yorickpeterse>
or anything close to "this is defined behaviour"
<yorickpeterse>
instead of a "happy little accident"
<brixen>
anyway, what is the question we're trying to resolve?
<yorickpeterse>
I actually don't know :P
<mustmodify>
he asked if you guaranteed that his value would be what he expected without mutexes.
<mustmodify>
and he said MRI promised that.
<mustmodify>
And I asked for documentation.
<brixen>
yorickpeterse: so, I was triaging old tickets yesterday and about 10% of them look like build system issues
<mustmodify>
because I remember working with threads in C and thinking that you could pretty much guarantee it would figure out how to screw you if it could.
<brixen>
yorickpeterse: one idea I was pondering is a VM in sh that we can target with a simple build language :)
<pchalupa>
the question is not about when it can switch, it's about: can you see different values in @ivars from two different threads ?
<yorickpeterse>
brixen: a lot of them are from the 2.0/2.1 days
<yorickpeterse>
2.0/2.1 was a bumpy ride
<brixen>
yorickpeterse: yeah, a lot of tickets are period
<yorickpeterse>
pchalupa: in rbx, yes
<brixen>
working on cutting them down
<brixen>
pchalupa: *nothing* in Ruby is guaranteed to have volatile semantics
|jemc| has joined #rubinius
<yorickpeterse>
pchalupa: since we have proper concurrency thread 1 could set an instance variable and then change it, while thread 2 at exactly the same time uses a variable
<brixen>
pchalupa: you'd need to use some specific facility to guarantee that
<yorickpeterse>
s/a/the variable
<brixen>
evan played with a way of annotating a class so that all ivars would have volatile semantics
<pchalupa>
yeah that is what I expect on Rubinius and JRuby
<brixen>
I should say, nothing in vanilla Ruby
<yorickpeterse>
brixen: VM in sh....coffee too strong? :P
<pchalupa>
my question os about MRI, so you need to synchronize there too?
<brixen>
yorickpeterse: hah, it was only a quad shot
<brixen>
pchalupa: dunno, we don't develop MRI
<brixen>
pchalupa: there are no docs that I know about MRI's memory model
<brixen>
but that's something you could go ask on the ruby-core mailing list
<yorickpeterse>
Worth keeping in mind: with LLVM 3.6 coming up we'll probably have more build problems coming up
<pchalupa>
sadly no :/
<yorickpeterse>
I believe 3.6 is supposed to be released in February
<brixen>
yorickpeterse: aww shit
<brixen>
best get crackin on that
<yorickpeterse>
Brace for all the distros that insta-upgrade (e.g. mine)
<brixen>
I'd be happy if some distros even had packages for 3.x
<pchalupa>
brixen let me backtrack, original question was about rubinius :) if you can do save immutable object in multi-threaded environment without locks?
<yorickpeterse>
brixen: Yeah, probably for a while we'll end up with 3.5 vs 3.6 #ifdef :/
<brixen>
pchalupa: could you rephrase that question, it doesn't make sense to me
<pchalupa>
brixen similar to final fields in Java,
<yorickpeterse>
unless we're going to require 3.6
<brixen>
yorickpeterse: the APIs in 3.6 are in 3.5, no?
<yorickpeterse>
Not sure if MCJIT is
<pchalupa>
brixen sorry s/save/safe/
* yorickpeterse
still needs to figure out if it's "msee-jit" or "MacJit"
<brixen>
pchalupa: there are no volatile semantics on Ruby ivars in Rubinius
<pchalupa>
brixen if you create an object with one final field in java you can pass it safely between threads.
<brixen>
pchalupa: if you want a memory barrier, use Atomic
<pchalupa>
I am thinking about following: def initialize; @a = 1; freeze; Rubunius.memory_barrier; end; then I should be fine passing that object around without further locking? I am correct or is it a stupid idea and I am missing something.
<brixen>
pchalupa: if the object is not mutated, there's no need to lock
<brixen>
if all you want is a memory barrier at a point in time, we could expose that independent of Atomic
<cremes>
not sure if all of the FFI syntax is compatible with rbx’s internal FFI, but i bet it’s close.
<|jemc|>
cremes: in places it's not, let's work to close the gap - that's one of my goals
<cremes>
and it uses libclang to do most of the heavy lifting.
<brixen>
cremes: cool
<cremes>
|jemc|: you knew about this and didn’t tell me? shame on you! :P
<|jemc|>
knew about what? that there are differences between the two APIs?
<cremes>
no, that there was a tool that you could feed a C header into and it would spit out FFI::Structs and stuff.
<|jemc|>
heh, either I misinterpreted
<|jemc|>
or you did
GitHub69 has joined #rubinius
<GitHub69>
[rubinius] brixen pushed 4 new commits to 1.8.7: http://git.io/FmpS
<GitHub69>
rubinius/1.8.7 0a101bd Brian Shirai: Fixed rubysl-readline version in Gemfile.installed.
<GitHub69>
rubinius/1.8.7 4cae271 Brian Shirai: Updated Changelog.
<GitHub69>
rubinius/1.8.7 1f3e6c8 Brian Shirai: Updated News.
GitHub69 has left #rubinius [#rubinius]
GitHub88 has joined #rubinius
<GitHub88>
[rubinius] brixen pushed 1 new commit to master: http://git.io/Fmhl
<GitHub88>
rubinius/master db75dd1 Brian Shirai: Removed 1.9 language syntax in configure.
GitHub88 has left #rubinius [#rubinius]
<yorickpeterse>
hmpf, seems I'll be writing a language detector this week
<yorickpeterse>
or alternatively trying to fix our current Java based one
<yorickpeterse>
brrrr
<yorickpeterse>
if I write more Java I might turn into a factory :<
<yorickpeterse>
(or a factory bean even)
GitHub170 has joined #rubinius
<GitHub170>
[rubinius] brixen tagged v1.4.1 at 1.8.7: http://git.io/FYfM
GitHub170 has left #rubinius [#rubinius]
<mustmodify>
I have another fun WTF-is-going-on-with-Rubinius issue. :( Using MRI 2.1.4 and rails 4.2, when I start my server, I see the app in all its glory. When I use rbx-2.4.1, I get "Routing Error: Missing or uninitialized constant: PagesController" ( this happens for every controller. ) WHen I go into the console and type PagesController, it works perfectly. I'm having a second issue that *may* be comorbid. When I start a server, it says, "Listening on tcp://
<brixen>
mustmodify: please try rbx 2.5.1
<brixen>
mustmodify: hot off the presses
<mustmodify>
ok
<mustmodify>
was there a 2.5.0 that I missed? Or are you guys taking "odd numbers are stable" super-seriously?
<|jemc|>
brixen: I'd honestly like to help out with getting the project past this roadblock and on to other things like MCJIT, but if I have time to devote to a massive chunk of shell scripting, it'll be at least few weeks from now
<cremes>
anyone know of any example where a built-in class (like Array as defined in array.cpp) calls from C++ code into Ruby?
<cremes>
and I’m not asking about the CAPI.
<cremes>
i’m interested in where a class calls *itself* from C++ to Ruby.
elia has joined #rubinius
<|jemc|>
cremes: looking for usage of Object::send in vm/builtin gives me just autoload.cpp
<|jemc|>
vm/builtin/autoload.cpp:26
<cremes>
i see it, thanks.
<cremes>
now i need to figure out how to pass args
<cremes>
here’s what i’m trying to do…
<|jemc|>
object.hpp:174 has the declaration with args
<|jemc|>
of type C++ "Array"
<cremes>
I want to replace IO::open_with_cloexec’s code to just call into the ruby method with the same name so it can do the work.
<cremes>
|jemc|: great! just what i need!
<|jemc|>
ruby-ize all the things!
<cremes>
that’s the plan.
<cremes>
we’ll see how much of it brixen accepts… :)
elia has quit [Quit: Computer has gone to sleep.]
nirvdrum has joined #rubinius
|jemc| has quit [Quit: WeeChat 1.0.1]
|jemc| has joined #rubinius
<yorickpeterse>
cremes: calling Ruby from C++ can lead to weird bootstrapping problems
<cremes>
yorickpeterse: discovering that… :)
<yorickpeterse>
e.g. the C++ code might run before the Ruby code has been set up
<cremes>
yorickpeterse: i see in ontology.cpp where all the built-in classes get initialized and started. where is the code that loads the ruby code in kernel/bootstrap and kernel/common?
<yorickpeterse>
there's some arcane magic and alcohol involved
<yorickpeterse>
and the rubinius-compiler
<yorickpeterse>
I don't remember specifically where it starts
<yorickpeterse>
don't hate me for the quality of some of the code, it's being worked on :P
<mustmodify>
machine learning. I read a cool book where there were machine-learning robots that could only be used for about an hour or they would figure out that you were smarter than them and take over.