<squeaky_pl>
I don't know if somebody reported this but vmprof.com seems to be broken. When I click on JIT button it says not found.
<idnar>
squeaky_pl: vmprof doesn't capture/upload JIT log by default, so maybe you're viewing a run without that?
<squeaky_pl>
idnar, yes you are right :-) So a little UX suggestion, just say that you need to pass that option and dont show 404 because it misled me.
<idnar>
ah, yes, indeed
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 246 seconds]
marky1991_2 has joined #pypy
antocuni has joined #pypy
<arigato>
pfff, fighting the Win32 API is not fun
adamholmberg has joined #pypy
<fijal>
arigato: what are you trying to do?
Rhy0lite has joined #pypy
marky1991_2 has quit [Ping timeout: 240 seconds]
<arigato>
CreateThread() seems to succeed, but the new thread never runs
marr has quit [Ping timeout: 255 seconds]
marr has joined #pypy
o11c has joined #pypy
<arigato>
ok, I should probably not use CreateThread() in the first place, but _beginthread() (of course, it makes sense)
<realitix>
hello arigato. I am finally selected for EuroPython. Look forward to see you there !
<antocuni>
realitix: cool, what will you talk about?
<realitix>
antocuni: I will talk about cffi
<realitix>
wednesday 10:30
<arigato>
:-)
<antocuni>
realitix: note that on the schedule they put your nickname instead of the real name
<antocuni>
not sure if this was intended or not
<realitix>
you're right I will ask them to write my real name
<arigato>
ok, no clue. neither CreateThread() nor _beginthread() appear to work in by case, and I have no idea what I did wrong
rokujyouhitoma has quit [Ping timeout: 260 seconds]
vkirilichev has quit [Remote host closed the connection]
Ryanar has joined #pypy
arigato has quit [Quit: Leaving]
oberstet has quit [Read error: Connection reset by peer]
aboudreault has quit [Excess Flood]
aboudreault has joined #pypy
ELFrederich has joined #pypy
<ELFrederich>
I have a C function I'm trying to create a wrapper for using cffi. One of its arguments is: `const char[WSO_name_size_c + 1] dataset_name` ... I'm trying to get this to compile in a cdef but it's not working. I tried putting a `#define WSO_name_size_c 64` in the cdef and tried putting `#define WSO_name_size_c ...` in there a well. I still get an error
arigato has joined #pypy
<antocuni>
ELFrederich: note that array sizes are basically ignored in C function declaration; you could just declare it as "const char*"
Remi_M has quit [Quit: See you!]
<ELFrederich>
antocuni: but my Python bindings have an opportunity to be smarter than my C bindings right?... I can actually check the length within my Python wrapper
<antocuni>
yeah, but you don't need to tell cffi about it
<ELFrederich>
antocuni: okay... I'm wodering then how my bindings can get this knowledge about the constants.
<antocuni>
I suppose there is no way to get this knowledge out of cffi
<ELFrederich>
antocuni: won't cffi need to know about these lengths when they're part of a struct?... to be able to allocate the appropriate memory?
<ELFrederich>
antocuni: okay... I'll do that in a separate step then. I'll have a C file which generates name: value for all the #defines
<idnar>
it depends on if you are allocating the struct yourself or just calling a library function that does it
<ELFrederich>
idnar: it'll be both. There are functions which take a struct as an argument
<ELFrederich>
idnar: and not a struct that was returned from another function, in C the API is that you create one of these structures yourself as input to the function
<arigato>
ELFrederich: it's a C quirk: the C declaration "int foo(int a[5])" is absolutely equivalent to "int foo(int *a)", but of course it's not the same in other contexts
oberstet has joined #pypy
<ELFrederich>
arigato: is it the same in a struct definition? There are structures that have like 9 members which are character arrays like `char object_name[WSO_name_size_c + 1];`
<arigato>
no, only in function arguments
<ELFrederich>
okay... this is a problem then; hmm
jcea has joined #pypy
<ELFrederich>
From the last time I tried converting from Cython to cffi I had a solution which used ffi.verify()... in there I was able to use `...` in a lot of places in my cdefs.
<lukasa>
ELFrederich: That remains largely true
<ELFrederich>
Here I'm trying to do ABI out-of-line
<antocuni>
ELFrederich: ffi.verify is API mode and you can use '...'; ABI mode does not support '...'
<lukasa>
As much as possible I recommend against doing ABI mode
<ELFrederich>
antocuni: right... and I can't use API mode because when I do and I package my application using PyInstaller it behaves weird
<ELFrederich>
ABI when bundled with PyInstaller seems to work fine
<antocuni>
I suggest to fix PyInstaller :)
<lukasa>
+1
<ELFrederich>
Can't really do that... too far down the rabbit hole
<lukasa>
"Note that some bundler tools that try to find all modules used by a project, like PyInstaller, will miss _cffi_backend in the out-of-line mode because your program contains no explicit import cffi or import _cffi_backend. You need to add _cffi_backend explicitly (as a “hidden import” in PyInstaller, but it can also be done more generally by adding the line import _cffi_backend in your main program)."
<ELFrederich>
I'm to the point where I may have to give up on calling this stuff from Python directly at all and just maintain a bunch of .c files that I call using subprocess.check_output
<ELFrederich>
lukasa: yup... I ran into that. But even when I did do a "import _cffi_backend" in my main program the application was behaving differently.
<lukasa>
What does "differently" mean here?
<lukasa>
Let's treat this like a bug
<lukasa>
What did you do, what did you expect to happen, what actually happened?
<ELFrederich>
okay... I have a binding to a function `int ITK_auto_login(void);` when I call it from C, or from cffi, or cython it'll just log in and return 0.
<ELFrederich>
when I call it from cffi in API mode the terminal will go interactive and prompt for a username and password
<ELFrederich>
sorry... cffi in API mode actually works fine, but when bundled with PyInstaller is when it goes interactive
<ELFrederich>
ABI mode works fine with or without PyInstaller
<lukasa>
Hmm, ok. Does your code link against a separate DLL?
<lukasa>
(Or .so, or .dylib)
<ELFrederich>
lukasa: .dll (unfortunately my dev environment for now is Windows)
<ELFrederich>
ITK_auto_login is implemented in a shared library
<lukasa>
So you're linking against a DLL. What are the odds that you're picking up different copies of the DLL, or accidentally not linking it correctly?
<lukasa>
Put another way: when you run in PyInstaller, are you loading the same code as when you run outside it?
<ELFrederich>
lukasa: I don't know what would influence that other than environment variables. I put at the top of my script `for k, v in sorted(os.environ.keys()); print(k, v)` ... when I ran it inside and outside of PyInstaller I got the same environment
Ryanar has quit [Quit: Ryanar]
<ELFrederich>
this was from the same terminal... so is there something else I could have checked besides environment variables?
<lukasa>
ELFrederich: Yeah, I'm afraid there is. =(
<lukasa>
Sadly, the question is what copy of the DLL is being picked up
<lukasa>
I am not sure how to ask Windows this question, but on (say) macOS you'd point `otool -L` at the binary to see what shared objects it loads
<ELFrederich>
lukasa: I think there is only one .dll on my system which has ITK_auto_login defined
Ryanar has joined #pypy
<lukasa>
ELFrederich: Yeah, but "i think" is not quite the same as "i know"
<lukasa>
Your only other option is to look into the definition of the code and see what triggers the interactive prompt
<antocuni>
ELFrederich: how do you launch the program without PyInstaller? From the command prompt? By calling python.exe or pythonw.exe?
<lukasa>
But it's hard to see how PyInstaller can be at fault here
<ELFrederich>
So if there is only one implementation of ITK_auto_login within libtcinit.dll and it goes interactive asking me for username/password ... that's telling me it found the correct/only one. Something else about the environment is weird causing the implementation to behave differently
<ELFrederich>
from python.exe
<ELFrederich>
maybe I need to spend 30 minutes and re-create this scenario since you guys are interested ;-)
<ELFrederich>
so API out-of-bounds is the best/suggested way to go right?
<ELFrederich>
I'll get an example working like that and then try to use PyInstaller
<lukasa>
Yeah, API out-of-band is definitely the best way to go
kanaka has quit [Ping timeout: 255 seconds]
rokujyouhitoma has joined #pypy
<ELFrederich>
what code should I put in the set_source vs the cdef?
<arigato>
in the simple cases, set_source() should contain only "#include <header_of_your_lib.h>"
<ELFrederich>
ahh... I think I found a Git branch of my work. Hopefully that's it
rokujyouhitoma has quit [Ping timeout: 255 seconds]
<lukasa>
ELFrederich: Yeah, that doesn't seem wrong. So I'd assume there is a problem inside the library you're linking such that it cannot find your credentials
<ELFrederich>
lukasa: the other problem: when I use ABI mode with PyInstaller for this trivial example.py I get a 5MB .exe, with API mode I get a 109MB exe
<ELFrederich>
actually... that was with directory mode, with a single .exe it's 36MB
<arigato>
uh, maybe with ABI mode it uses the DLL and with API mode it links the whole code in, not using any DLL?
marky1991 has joined #pypy
<arigato>
how large are the _*.pyd generated by the *_build.py files?
<ELFrederich>
arigato: let me look
<ELFrederich>
pyd files at 12 and 16k
adeohluwa has joined #pypy
<arigato>
why does pyinstaller produce 36MB??
<ronny>
i very vaugely recall it includes python and the stdlib into the installable element
<antocuni>
is it possible that pyinstaller copies the DLLs inside the final exe?
yuyichao has quit [Ping timeout: 240 seconds]
<arigato>
ronny: right, but even that should not be 36MB
<arigato>
we're trying to understand why if you use cffi in API mode you get a 36MB exe, and in ABI mode you get a 5MB exe---which makes more sense
<antocuni>
a quick googling reveals that pyinstaller has options to include all the required DLLs in the final exe
<ELFrederich>
let me verify this again...
<arigato>
I guess 36MB is the compressed version of the 109MB you get in directory mode
<antocuni>
so my theory is that with ABI mode, it obviously doesn't know about the extra DLL (because it will dlopen()ed only at runtime) and thus does not include it
<arigato>
so the real question is, what is in these directories
<arigato>
ah
<antocuni>
with API mode, the pyd files are linked against the DLL, so pyinstaller decides to copy them inside
<arigato>
ah!
<kenaan>
stefanor default ad6889740dc0 /rpython/translator/platform/__init__.py: Some more (obscure) architectures that would like -fPIC
<arigato>
so you end up with a copy of the DLL you're calling
<antocuni>
and I also bet that the fact that it's a *copy* of the DLL triggers the difference in ITK_auto_login
<arigato>
yes
<antocuni>
but that's just a theory of course :)
<arigato>
it's a good theory
<arigato>
the DLL gets decompressed and put in a temp dir
aboudreault has quit [Excess Flood]
<arigato>
and loaded from there
<antocuni>
yes, something like that
<arigato>
so some paths are different
<arigato>
is there an option to PyInstaller to not copy DLLs?
realitix has quit [Quit: Leaving]
<arigato>
(that's also a good explanation for why the same problem exists wth Cython instead of CFFI-API-mode)
<antocuni>
ah but IIRC ELFrederich said that with cython it worked fine; this goes against my theory of course
<kenaan>
arigo cffi/cffi b0f094e3f3eb /testing/embedding/: Basic "does not crash" test for the error logic in _cffi_initialize_python
<kenaan>
arigo cffi/cffi cff66142f09b /: Windows: when a CFFI embedding attempt fails, attempt to open a non-modal MessageBox() in addition to sending the...
adamholm_ has joined #pypy
adamholmberg has quit [Ping timeout: 240 seconds]
rmesta has quit [Quit: Leaving.]
rmesta has joined #pypy
vkirilichev has quit [Remote host closed the connection]
yuyichao has joined #pypy
yuyichao has quit [Ping timeout: 246 seconds]
vkirilichev has joined #pypy
black_ant has quit [Ping timeout: 240 seconds]
rmesta has quit [Quit: Leaving.]
rmesta has joined #pypy
rmesta has quit [Client Quit]
rmesta has joined #pypy
rmesta has quit [Client Quit]
rmesta has joined #pypy
rmesta has quit [Client Quit]
rmesta has joined #pypy
rmesta has quit [Client Quit]
rmesta has joined #pypy
rokujyouhitoma has joined #pypy
rmesta has quit [Client Quit]
rmesta has joined #pypy
rmesta has quit [Client Quit]
rmesta has joined #pypy
rmesta has quit [Client Quit]
rmesta has joined #pypy
rmesta has quit [Client Quit]
rmesta has joined #pypy
antocuni has quit [Ping timeout: 255 seconds]
rmesta has quit [Client Quit]
rokujyouhitoma has quit [Ping timeout: 240 seconds]
adamholm_ has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 246 seconds]
adamholmberg has joined #pypy
black_ant has joined #pypy
vkirilichev has quit [Remote host closed the connection]
black_ant has quit [Excess Flood]
vkirilichev has joined #pypy
black_ant has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholm_ has joined #pypy
adamholmberg has quit [Ping timeout: 240 seconds]
<nanonyme>
arigato, out of curiosity: is the message box suppressable?
<arigato>
nanonyme: not directly, but it's also tweaked to not be modal (so not blocking the app, and disappearing if the program closes)
rokujyouhitoma has quit [Ping timeout: 268 seconds]
<nanonyme>
Right
<arigato>
I could add yet another API to disable it, that would be in line with the Windows API style---but I don't like that :-)
pilne has joined #pypy
<nanonyme>
Just a question. I guess it's fine as is. I've specifically been suffering of hanging Windows service processes lately (nothing to do with CFFI) so started wondering
<arigato>
:-/
<arigato>
I hope I took enough care so that the MessageBox() cannot block
<nanonyme>
Turns out eg asserting in C code inside a Windows service is *bad*
<nanonyme>
It will permanently hang the process
<arigato>
it's starting a new thread, and calling MessageBox() there; and as long as there is such a MessageBox thread then further attempts to open one are ignored
<arigato>
bah :-(
<nanonyme>
Or in details it opens a dialog in system session that must be reacted to and no one sees
<arigato>
that's because I hate this aspect of Windows that I tried to make it slightly more useful