<kenaan>
arigo py3.5 45f61802dc1c /lib_pypy/_lzma.py: More GC tweaks. Trying out to call add_memory_pressure() with a negative number. I *think* it should always give wh...
cstratak has joined #pypy
cstratak has quit [Client Quit]
nimaje is now known as Guest30163
Guest30163 has quit [Killed (card.freenode.net (Nickname regained by services))]
nimaje has joined #pypy
antocuni has quit [Ping timeout: 260 seconds]
jamesaxl has quit [Read error: Connection reset by peer]
<nanonyme>
fijal, won't we have fun times when people start relying on that property
<fijal>
they will
<nanonyme>
Yeah, and then all future implementations of Python dict will need to provide that property whether efficient or not
<fijal>
maybe not a terrible property
<tos9>
that's ok it's 2017 who still uses `dict` anymore
<fijal>
I mean, people relied on dictionary order before
<nedbat>
fijal: it would be better to ask programmers to say they need ordered dicts, in my opinion, but that ship has sailed.
<nanonyme>
fijal, they did? What for
oberstet has joined #pypy
<LarstiQ>
nanonyme: all kinds of things
<LarstiQ>
nanonyme: a simple one, tests failing on .keys() = somelist comparison
TheAdversary has quit [Quit: Going down for maintenance.]
<nedbat>
LarstiQ: accidentally relying on arbitrary ordering is different than saying, "Dicts are ordered, I will use that"
<arigato>
imho it replaces things that work accidentally with things that work by design
<nedbat>
arigato: i wish we had just had people use ordereddict if they wanted an ordereddict.
<arigato>
a bit like "x is 5", which works accidentally as long as 'x' is an 'int'
<fijal>
nedbat: well, that *aslo* means you grow another builtin type for no reason at all
<nedbat>
fijal: the reason is so that in five years when we think of yet another dict optimization, we have the flexibility to implement it.
<fijal>
I don't think future potential optimizations are a good reason to grow already massive language tbh
<arigato>
and I think that the past 25 years have shown there is no important missing dict optimization
<fijal>
if anything, having tons and tons of builtin types is bad for implementing optimizations
<nedbat>
fijal: we already have dict and ordereddict? No need to grow anything.
<nedbat>
arigato: we might have said that 5 years ago about the previous 20.
<fijal>
did you know that OrderedDict is hard to implement using regular ordered dict because they added new methods on it?
<fijal>
(for example)
<fijal>
there are also slight differences in semantics
<fijal>
so I would vastly prefer "never use OrderedDict" and we can focus on optimising a smaller language
<arigato>
e.g. what occurs if you change an OrderedDict while iterating over it
<nedbat>
i'm glad you guys are happy with it. i would have tried to make the central data structure have fewer contraints, and therefore more implementor flexibility.
<fijal>
nedbat: as I said, the main problems we have are around the amount of shit there is (bytearray, str, memoryview, buffers more than one of them) etc.
<nedbat>
fijal: ok, good to hear.
<fijal>
*and* the constraint seems to be "has a C API that looks like this"
TheAdversary has joined #pypy
<fijal>
which is a much more serious constrain than ordering dicts and the one I would be hapy to remove
slackyy has quit [Read error: Connection reset by peer]
<arigato>
that's not necessarily true e.g. on the JVM or .NET
gabriel-m has joined #pypy
<simpson>
I think that the problem is that for many folks, ordered dicts are obviously better and desirable, and not having them earlier was a language flaw.
<nanonyme>
arigato, yeah, I guess so
<nanonyme>
I guess it would also be fair dict memory would be lazily decreased and rehashing would happen due to memory pressure
<nanonyme>
Which if done might lead to rehashing to happen if storing they keys as a list would hit a memory usage boundary
<nanonyme>
s/they/the/
<nanonyme>
(maybe by eg dict telling memory manager that it should be compacted later)
<nanonyme>
Anyway, yeah, I guess even my constraint was bad
<nanonyme>
Hm, I think it's really sad PyPy cpyext improvals resulted in lxml-cffi basically dying out and now the only living thing is the old lxml which uses cython
<cfbolz>
eh
<cfbolz>
nanonyme: the only sustainable thing would have been lxml switching to the cffi version, honestly. I suppose that was unlikely?
<nanonyme>
cfbolz, the incentive to switch disappeared with cpyext improvements
<nanonyme>
Admittedly it wasn't huge but it completely disappeared
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<tos9>
nanonyme: it was never gonna happen, the lxml author is the cython author
<simpson>
And not the sort of person who takes being told that everything is terrible in a good way.
marr has quit [Ping timeout: 255 seconds]
* simpson
contributed a tiny bit to Cython long ago
<njs>
lzma (the algorithm) likes to use *tons* of memory, it's both part of why it compresses so well and also a serious problem when you don't know how many resources are available to the recipient of your compressed file.
<njs>
thanks for subscribing to random compression algorithm facts
* LarstiQ
eagerly awaits the next installment
jamesaxl has quit [Quit: WeeChat 1.7.1]
<tos9>
njs: That was nice, but could be shorter, next time maybe think about how you can say more things in fewer words
<simpson>
njs: This was a factor in an actual use-case I had a few years ago, where I wanted to make some backups of Wii discs, and the format that they come in is highly compressible but the CPU time at both ends was cutting into I/O time. I profiled and chose gzip over bz2, lzma, and xz because it had the best ratio. Go figure~
<njs>
tos9: I think the spirit of compression algorithm facts would be more, saying things very redundantly and letting the algorithm worry about optimizing it
gabriel-m has joined #pypy
<njs>
simpson: yeah, the compress-once-decompress-once case is really not what most of these systems are optimized for...
<simpson>
njs: Well, the thing was, this was the final on-disk format, but it would have taken a week instead of a night if I had chosen xz instead of gzip, and for only a projected 5% disk savings, which wasn't worth it because Wii discs have so much empty and patterned space that they compress *really* well.