sb0 changed the topic of #m-labs to: ARTIQ, Migen, MiSoC, Mixxeo & other M-Labs projects :: fka #milkymist :: Logs http://irclog.whitequark.org/m-labs
kaalia has quit [Ping timeout: 240 seconds]
kaalia has joined #m-labs
<sb0>
cr1901_modern, yeah. but the new system is in place now.
<sb0>
lol, eetasia is running an infomercial "Examining the most underrated FPGA design tool ever" about simulink
<sb0>
"underrated"
<sb0>
some people have incredibly low standards
<sb0>
or maybe, reality distortion field
<cr1901_modern>
I'm a MATLAB apologist, but simulink is garbage
<cr1901_modern>
Python should be an acceptable replacement soon enough. I do know I'll be using exclusively after Master's is done.
<cr1901_modern>
sb0: I'll submit a patch to fix Windows testing later tonight, and take a look at the naming problem. In the meantime, while I have some time- is VPI support something like: >>
<whitequark>
in fact my first sputterer ever was based on that design
<whitequark>
I gave up on it because 1) the anode heats up a LOT and the epoxy softens, 2) oxygen messes stuff up and I want argon there
<whitequark>
both of which don't play well with glass jars
<sb0>
cr1901_modern, yes, but signals go both ways, and time should be dealt with appropriately
<cr1901_modern>
oh right, because the next event to occur could be within an instance module
<cr1901_modern>
Interesting video. Wish I had space to do something like that. But I don't have any application in mind either
<sb0>
cr1901_modern, there are many problems; i'd look at how myhdl does it
<cr1901_modern>
"many problems" doesn't tell me much. I'm looking at the myHDL source now just to get an idea.
<sb0>
well, the fundamental issue is to run the verilog simulator for the correct amount of time between calls to python (iterations of the main loop in Simulator.run())
<sb0>
there is the problem of delta cycles, and that of the timescale correspondence
<sb0>
some details like reset
<sb0>
and various implementation issues
<sb0>
speaking of, if we can trash sockets for a less sucky IPC system, that would be good
<sb0>
but IPC in 2015 still does not work
<cr1901_modern>
Named Pipes is probably best on Windows, but Python doesn't support them well
<cr1901_modern>
it's the closest to SOCK_SEQPACKET
<sb0>
capturing stdin/stdout works pretty well in this case, but requires good control of what the child process uses them for
<sb0>
if an instance uses $display, that can break
<whitequark>
sb0: dbus? :]
<sb0>
on windows?
<cr1901_modern>
Not on windows
<whitequark>
there's a port...
<whitequark>
... but you probably don't want to use it.
<sb0>
yeah, if only for dependency hell
<cr1901_modern>
Perhaps Migen sadly needs a light platform wrapper
<sb0>
what do you mean?
<cr1901_modern>
in other words, Windows vs everyone else (big surprise there)
<whitequark>
sb0: there's zmq.
<sb0>
bah, zmq is just wrapped sockets
<whitequark>
I wouldn't say so
<cr1901_modern>
i.e. migen.wrapper.IPC.create_pipe(), migen.wrapper.IPC.send_data_over_pipe()
<whitequark>
zmq is message-oriented and provides sequencing and guaranteed delivery
<cr1901_modern>
Contrived example
<cr1901_modern>
It's LGPL
<sb0>
yeah, you can do that by opening a TCP socket and doing loop: length = read(4), message = read(length)
<sb0>
not worth importing a large library for that
<whitequark>
I guess if you just want a named pipe, it's overkill
<whitequark>
can't Python use those on Windows, anyway?
<sb0>
also, my main issue with sockets is there is no clean and portable way to connect them between parent and child processes
<sb0>
and zmq does not help with that
<cr1901_modern>
whitequark: Not easily. When I added sim support to Windows, trying to support named pipes required importing the C bindings to Windows dlls
<whitequark>
what's wrong with: reconnecting to a known endpoint after fork, recording pid in parent, and sending pid in child when messaging?
<sb0>
you cannot fork on windows
<whitequark>
doesn't matter, there is no data shared
<whitequark>
and CreateProcess gives you child pid alright
<whitequark>
s/fork/spawn/
<sb0>
another problem is the "known endpoint"
<sb0>
TCP ports and unix sockets on the filesystem are globally on a machine and not specific to a parent/child process relationship
<sb0>
*global
<cr1901_modern>
Idk how to connect stdin of a child process to stdout of a parent on Windows
<cr1901_modern>
if it can be done
<whitequark>
you can provide the standard handles to CreateProcess
<whitequark>
I'm reasonably certain Python has code for that already
<sb0>
and still, from a security/isolation perspective, anything can connect to your socket, not just the process you intended
<whitequark>
sb0: allocate an ephemeral port and set an environment variable?
<whitequark>
hrm
<sb0>
the proper solution is to be able to create as many stdin/out as you want and redirect them, but no major operating system supports that
<whitequark>
ok, another idea
<cr1901_modern>
sb0: Do what FTP passive mode does
<whitequark>
you can redirect a single stdout, right?
<whitequark>
on startup, hijack the stdout, create all the stdouts you want, then multiplex
<sb0>
yes, stdin/out redirection is fine (and is what artiq uses all over the place) but requires good control of the child process
<sb0>
which is fine when you are writing it, but can be problematic when it's a 3rd party verilog simulator
<cr1901_modern>
whitequark: Does that handle $display?
<whitequark>
then you just have to handle multiplexing at your perimeter.
<sb0>
if you hijack $display, yes
<whitequark>
sb0: when would my solution not work?
<sb0>
I'm not sure if you can hijack $display by loading a VPI module into a verilog simulator
<whitequark>
why do you need to hijack $display ?
<sb0>
and more generally what normally goes to stdout/stderr in the simulator
<whitequark>
or rather, what behavior are you trying to get from verilog simulator?
<sb0>
because otherwise the normal output of the simulator, which comes from $display and other places, will mess up your IPC protocol
<sb0>
so you want to encapsulate it into the IPC protocol
<sb0>
and VPI module loading might not be powerful enough to allow that
<whitequark>
oh, so you want to do IPC output from a module loaded into the simulator.
<whitequark>
what is VPI?
<whitequark>
is it a binary?
<sb0>
a standard mechanism for loading plugins into verilog simulators
<whitequark>
or rather, is it a binary that executes in the same AS as the simulator?
<sb0>
it's a shared object on linux, so yes, you can perhaps play some tricks with the dynamic linker
<whitequark>
nah
<whitequark>
dup2()
<sb0>
but I said "portable" above
<whitequark>
and whatever the fuck is the Windows equivalent of dup2()
<whitequark>
I know for a fact there is one, I used it
<whitequark>
so you can hijack the output from inside your VPI.
<whitequark>
assuming you don't want that, there is clearly another more generic way
<sb0>
oh, interesting
<cr1901_modern>
Well, if we accept that the C/whatever code implement the VPI is going to have #ifdef _WIN32s, that should work
<cr1901_modern>
that;s going to implement the VPI*
<sb0>
so the issue now is to scrap whatever output the simulator produces before initializing VPI, but it should be solvable with magic string
<whitequark>
include a long, unique synchronization word before every packet from VPI. which will work independently of your environment, assuming you can get the packet from VPI in one piece
<whitequark>
then, on the demultiplexor side, take all raw input and put it 0th stream. and take everything else and put it in appropriate nth stream.
<whitequark>
if you use unbuffered write/WriteFile &c APIs, and the simulator is not multi-threaded, that will also work
<whitequark>
this is, frankly, perverse
<cr1901_modern>
the generated Verilog module is just going to be all the Instance modules laid out in a singlee file with all relevant connections, correct?
<sb0>
yeah, it's perverse, see my comment about the state of IPC
antgreen has quit [Write error: Connection reset by peer]
antgreen` has joined #m-labs
<sb0>
cr1901_modern, there won't be any generated verilog
<sb0>
the only reason you want an Instance is because you have the verilog source already somewhere
<sb0>
I guess you can load multiple verilog modules into one simulator. if not, run several simulators. this is one of the many problems I mentioned earlier...
<sb0>
what does myhdl do?
<sb0>
actually maybe you need to generate some wrapper that sets the parameters
<sb0>
if they cannot be passed on the command line. i don't know.
<cr1901_modern>
If there is no generated verilog, how will Iverilog or a verilog sim be told to "transmit data back to Migen"?
<cr1901_modern>
Let me get back to you re: myHDL in a few minutes
<sb0>
I don't know, you may need some verilog stubs. depends what the VPI API provides to you.
<sb0>
the old simulator uses a stub, but that was not a problem as there was already temporary files and all. but ditching the temporary files and doing all from VPI would be good.
<cr1901_modern>
According to this bug report, $display cannot be hijacked in iverilog. The bug is unrelated; it's the fact that vvp will refuse to simulate if system.vpi isn't found.
<cr1901_modern>
Well, I suppose one could make their own system.vpi, but sounds like an absolutely terrible idea
<whitequark>
why? it's literally the same as pipe() and dup2() in unix
<whitequark>
ok, well, it's a strict superset
<sb0>
yes, but with pipe you need to do:
<whitequark>
oh, you said callbacks
<whitequark>
you can use IOCP to simplify that
<sb0>
create a pipe, pass one end to dup2, pass the other one to a reading thread that encapsulates into the IPC protocol
<whitequark>
well, yeah
<whitequark>
it's not conceptually any different on windows
<whitequark>
in case of the VPI you will need to spawn a thread, yes
<sb0>
thus: bloat, synchronization, shutdown issues, etc.
<whitequark>
a file is a kernel object, so there is no way around that
<sb0>
yeah, operating systems suck
<cr1901_modern>
sb0: I do not see us getting away without including at least one verilog stub that drives the clock
<whitequark>
meh. I don't see any problems with this
<sb0>
cr1901_modern, VPI can drive a clock just fine, so you seem quite confused
<whitequark>
"bloat", omg, 8k for a thread. such waste. threads are cheap
<whitequark>
the key point of kernel objects is they allow preemption on calls, and this is exactly what you want, unless you are OK with everything working under the same one runtime like .net
<cr1901_modern>
sb0: Great thing is, after rereading that I have no idea what I was trying to say either, so nevermind
<sb0>
mh. sync words or even, well, those shitty sockets might be a lesser evil than this mess
<sb0>
hm
<whitequark>
the question with sync words is, are writes atomic?
<whitequark>
sb0: wait, I completely forgot, Windows has LPOVERLAPPED
<whitequark>
you can, in fact, tell the kernel to call you, though it's still asynchronous pull and not a push API
<sb0>
pipe+fork does what i want, but not portably
<whitequark>
just use threads.
<whitequark>
seriously, it's not like it's that hard. use select() on *nix and WFMO on Windows
<whitequark>
should I write that for you?
<sb0>
you can't use select, the main program loop is that of iverilog
<whitequark>
sure
<whitequark>
spawn a thread and use a select there, waiting for stdout
<sb0>
or fuck windows and use pipe+fork
<whitequark>
I'm still waiting for an *actual* reason to not spawn a thread, instead of FUD
<sb0>
synchronization, shutdown
<whitequark>
a single mutex protecting addition of new streams, and none necessary
<cr1901_modern>
Let's not say "fuck windows" just yet. I kinda would like to simulate Migen modules that rely on other code. In any case, myHDL does the following to implement Verilog simulation: http://docs.myhdl.org/en/latest/manual/cosimulation.html >>
<cr1901_modern>
It's similar to what Migen did previously, although for whatever reason, in the linked example, they're assuming all the HDL code is Verilog and myHDL is simply being used for Unit testing
<cr1901_modern>
as opposed to having a mixed myHDL and Verilog design. So not really helpful
<sb0>
so there would be two pipes, one that gets connected to the old stdout via dup2, and the other one to send commands to the thread
<sb0>
the threads takes both streams via select, and multiplexes them onto the actual stdout
<whitequark>
sb0: I don't think there is a problem with stdin
<whitequark>
the thread selects on old-stdout, old-stdin and forward pipes
<sb0>
the thread should not need to touch stdin
<cr1901_modern>
actuallt nevermind, I think this is the reverse problem, please disregard the link
<sb0>
I don't think we need to support perverse verilog designs that take user input on stdin
<whitequark>
that simplifies it.
<cr1901_modern>
It's probably the easiest to take a route similar to myHDL, where the VPI contains functions that will bind inputs and outputs to Migen Simulator's signals. The Migen Simulator manipulates these values, including the clock, and vvp will be able to detect signal changes. >>
<cr1901_modern>
timescales will need to be synchronized somehow
nicksydney has quit [Quit: No Ping reply in 180 seconds.]
nicksydney has joined #m-labs
<sb0>
cr1901_modern, unit test or FHDL code interpreted in python, from the verilog simulator's perspective it's the same
ylamarre has quit [Quit: ylamarre]
<GitHub121>
[migen] sbourdeauducq pushed 1 new commit to new: http://git.io/vnEj9
<GitHub121>
migen/new 1857ec6 Sebastien Bourdeauducq: fhdl/namer: support ClockSignal and ResetSignal. Closes #24
<cr1901_modern>
sb0: (took a nap) myHDL uses file descriptors, so you and whitequark independently came up with the right idea fwiw
<sb0>
yes, but the problem is transferring those to the child process
<cr1901_modern>
Ahhh, nice catch. I didn't see that.
<sb0>
which can be done using fork on linux, apparently with this other method as well that I didn't know about, and it is compatible with windows using this msvcrt trick
<cr1901_modern>
I don't remember os.pipe being available on Windows, yet the docs say it is
<whitequark>
msvcrt trick?
<whitequark>
oh
<whitequark>
oh, interesting
<cr1901_modern>
Does os.pipe call mkfifo before returning?
<sb0>
anyone has a clue what is wrong with travis?
<whitequark>
Python on travis seems to be particularly fragile for some reason
<whitequark>
I've never seen so many bogus build failures on any project I've been on
<whitequark>
and no, I have no idea
<sb0>
hm, I can reproduce it actually, it's due to anaconda, not travis
<sb0>
and/or setuptools
<sb0>
the problem comes from this crap installing an empty /home/sb/anaconda3/lib/python3.4/site-packages/migen/__init__.py instead of the file I provided
<sb0>
ah. it checks out the master branch.
<sb0>
bah
<GitHub100>
[migen] sbourdeauducq pushed 2 new commits to new: http://git.io/vnuyP
<GitHub100>
migen/new 88f9d72 Sebastien Bourdeauducq: conda: use new branch (revert this after merge)
<cr1901_modern>
sb0: Does the Python simulator have any facilities for debugging combinational loops yet?
<cr1901_modern>
I guess I could do a call stack trace if necessary (since it eventually chokes with "recursion depth exceeded", but not really in an immediate position to fire up pdb
<sb0>
what? comb propagation does not use recursion
<cr1901_modern>
Erm, I think this is a simulator bug. Will post a gist in a minute
<cr1901_modern>
I got rid of all combinational logic in the test module, and the simulator still chokes with "recursion depth exceeded"