ChanServ changed the topic of #picolisp to: PicoLisp language | Channel Log: | Picolisp latest found at | check also for more information
schr0dinger has quit [Quit: Leaving]
schr0dinger has joined #picolisp
schr0dinger has quit [Client Quit]
<viaken> Regenaxer: Can I just say, picolisp is truly impressive? You've create a system that seems optimized to get stuff done in. I've not fully grokked it yet, but thank you for making it all the same.
<viaken> By the by, is the code/basis for the wiki available somewhere?
libertas has quit [Ping timeout: 240 seconds]
libertas has joined #picolisp
_whitelogger has joined #picolisp
rgrau has quit [Ping timeout: 240 seconds]
libertas has quit [Ping timeout: 240 seconds]
libertas has joined #picolisp
_whitelogger has joined #picolisp
_whitelogger has joined #picolisp
<Regenaxer> viaken, thanks! :)
<Regenaxer> and, yes, it can be downloaded at
<beneroth> Good morning viaken, Regenaxer
<Regenaxer> Good morning beneroth
<viaken> Technically I guess you're right. Morning, beneroth.
<beneroth> you're talking about Regenaxers approach to software design or about the download?
<viaken> With "Technically..."? No, about it being morning.
<viaken> Nearly 3AM here.
<Regenaxer> ah :)
<beneroth> ah
<beneroth> :D
<beneroth> you can't sleep? or not in the mood?
<beneroth> here it is nearly 9 AM
<viaken> Not been able to sleep, for reasons unknown. Possibly too much caffeine.
mtsd has joined #picolisp
<beneroth> happened to me too.
<beneroth> calming tea may help.
<beneroth> Morning mtsd
<mtsd> Good morning beneroth
<mtsd> calming tea? Sounds good
<clacke[m]> > viaken: Not been able to sleep
<beneroth> hey clacke[m]
<clacke[m]> First misread name as "vaken", meaning "awake" in Swedish
<beneroth> :D
<beneroth> so it is because of the name.
<viaken> heheh
<clacke[m]> hey beneroth
<viaken> Actually, I have some calming tea. That's not a bad idea.
<clacke[m]> enjoying time off this week
<clacke[m]> parents visiting hk
<beneroth> oh nice
<clacke[m]> chamomile tea is a nice calming tea
<mtsd> I like that too
<beneroth> or Lemon balm
<viaken> Those are both nice. This is Twinings Nightly Calm, and it's got a whole slew of things in it.
<viaken> Chamomile primarily, though.
<tankfeeder> morning
<beneroth> Morning tankfeeder
coffeecup12345 has joined #picolisp
<Regenaxer> Hi coffeecup12345!
<beneroth> Welcome coffeecup12345
coffeecup54321 has joined #picolisp
coffeecup54321 has quit [Remote host closed the connection]
coffeecup54321 has joined #picolisp
coffeecup54321 has quit [Client Quit]
coffeecup54321 has joined #picolisp
coffeecup54321 has quit [Client Quit]
coffeecup54321 has joined #picolisp
coffeecup54321 has quit [Client Quit]
<tankfeeder> new version
<tankfeeder> reading only words from file:
<Regenaxer> Wow, much simpler
<Regenaxer> well, only the parser
<tankfeeder> is it ok while-char ?
<Regenaxer> yes, fine
<tankfeeder> next hash on words and idx
<Regenaxer> ok
coffeecup12345 has quit [Remote host closed the connection]
<tankfeeder> memoization on cache!
<Regenaxer> Why the 'wait'?
<Regenaxer> hmm, I don't get it ;)
<Regenaxer> testing the 'hash' function?
<tankfeeder> test that works
<tankfeeder> first second, second time from cache :)
<tankfeeder> first 1 second wait, second time from cache
<tankfeeder> i will remove wait
<tankfeeder> hash on word will be a key from idx tree
<tankfeeder> for idx tree
coffeecup12345 has joined #picolisp
<Regenaxer> ok
<tankfeeder> now all works fast as python
alexshendi has joined #picolisp
<tankfeeder> but numbers are different
<Regenaxer> Great!
<tankfeeder> python:
<Regenaxer> oh
<tankfeeder> ('the', 41036), ('of', 19946), ('and', 14940)
<tankfeeder> picolisp:
<tankfeeder> (35280 "the" 40720) (56449 "of" 19916) (3201 "and" 14799)
<tankfeeder> so 41036 vs. 40720
<Regenaxer> different parsing?
<tankfeeder> yeah, unknown
<tankfeeder> even was word: 8619 vs. 8609
<tankfeeder> "was" word
<tankfeeder> # cat 135-0.txt | grep -ci was
<tankfeeder> 7787
<tankfeeder> the main problem where to stop and count task solved.
rob_w has joined #picolisp
<m_mans> Hi all!
<beneroth> hi m_mans
<Regenaxer> Hi m_mans!
<m_mans> My first day as PicoLisp programmer :)
<Regenaxer> Congratulations!!
<m_mans> does httpGate have any debug mode?
<m_mans> thanks! :)
<Regenaxer> hmm, 'ssl' has, with +
<Regenaxer> but not httpGate iirc
<m_mans> what about adding something similar (non-forking mode with more logging to STDOUT) ? :)
<Regenaxer> A quick way is to edit src/httpGate.c temporarily
<Regenaxer> That's how I did it
<Regenaxer> What do you look for?
<m_mans> yes, I'll maybe do that. I'm trying to start httpGate with one entry in the config file
<Regenaxer> just a single entry is fine
<m_mans> it works if I go to IP/<port>, but doesn't if I try IP/<appname>
<Regenaxer> ah
<Regenaxer> strange
<m_mans> configurator 8099 mmamkin /home/mmamkin/configurator ../log/configurator.log ../picoLisp/pil app/main.l @lib/app.l -main -go -wait
<Regenaxer> config format wrong?
<m_mans> something illegal?
<Regenaxer> no, seems correct
<m_mans> I'll investigate more
<Regenaxer> ok
<Regenaxer> Perhaps just strace?
<m_mans> how are the things with your ERP project?
<m_mans> ah, thanks! I've forgotten strace
<Regenaxer> good, though not much ERP currently
<Regenaxer> I'm doing things called "Simple Links" ;)
<Regenaxer> for several projects
<m_mans> what is it?
<Regenaxer> it is something for logistics, involving DB and mobile apps
<Regenaxer> In fact, it is nothing secret
<Regenaxer> if lucky, it will become a standard in logistics
<m_mans> hm, btw, it seems '#' is illegal in config file?
<Regenaxer> Perhaps '#' only in the first column allowed
<Regenaxer> I don't remember exactly
<tankfeeder> let is be like:
<Regenaxer> tankfeeder, I don't understand why hash+ is needed
<tankfeeder> create hash on word
<Regenaxer> yes, but 'cache' already hashes
<Regenaxer> so just store the word?
<Regenaxer> then also no idx is needed
<tankfeeder> hm
<Regenaxer> just the words in the tree
<Regenaxer> and the count could be stored there
<Regenaxer> as words are transient symbols
<Regenaxer> (but, sorry, I did not study the task)
<tankfeeder> my idx store in list: (<hash> "word" <counter>)
<tankfeeder> i sort on counter and output the most popular words.
<Regenaxer> yeah, but 'cache' already does something similar
<Regenaxer> hashes and makes an idx tree
<Regenaxer> So you just need to store the words
<Regenaxer> and I would put a count *into* the word
<Regenaxer> easier perhaps
<tankfeeder> ("the" . 1234) ?
<tankfeeder> or "the1234" :)
<Regenaxer> yes
<Regenaxer> but you could also (by val sort ...
<Regenaxer> kind of
<tankfeeder> damn
<Regenaxer> I think we have an example for that
<tankfeeder> where ?
<Regenaxer> let me search a little
<beneroth> Regenaxer, just glimpsed over the whitepaper. it has bold annotations in it xD
<beneroth> Regenaxer, will read it more thoroughly soonish (probably in a few days)
* beneroth worked for two retailers who have their own logistics, in one of them beneroth was part of the team who designed all logistic processes
<Regenaxer> sorry, was on phone
<Regenaxer> tankfeeder, I don't find the example I had in mind
<Regenaxer> but I think you know what I mean
<Regenaxer> a kind of modified cache
<Regenaxer> beneroth, what do you mean with bold annotations?
<Regenaxer> tankfeeder, what if you do it just with 'idx', and without 'hash'
<Regenaxer> Not balanced enough?
<Regenaxer> You could do for each "word":
<Regenaxer> (if (idx 'Idx "word" T) (inc "word") (set "word" 1))
<tankfeeder> ok
<Regenaxer> Not perfectly balanced, depending on how random the ordering of words is
<Regenaxer> But for a non-sorted input it is probably ok
<Regenaxer> Perhaps even faster, as less call overhead
<beneroth> Regenaxer, big black speech bubbles commenting the content
<Regenaxer> yeah, a bit unreadable
<beneroth> who is the main head behind Simple Links? You?
<Regenaxer> It is Ruediger Hagedorn of Consumer Goods Forum
<beneroth> ah nice
<Regenaxer> He is sometimes here, nick "rudha"
<Regenaxer> It was he who just phoned me ;)
<Regenaxer> coffeecup12345 here?
<Regenaxer> coffeecup12345 is in fact also involved :)
<beneroth> I hope you can establish a standard implementation and not just a standard definition. FOSS usually does standard implementation. Enterprise & Industry usually does standard specifications - then you have the problem of loopholes and of competing faulty implementations
<Regenaxer> Well, I just have a reference implementation
<beneroth> more is not possible. every vendor wants to sell his own shit. ^^
<Regenaxer> every company has to implement it on their own systems
<Regenaxer> (but everybody is free just to use my pil version of course)
<Regenaxer> modify it
<beneroth> depending on the licensing (this is to be considered! both the license for the standard documentation and the implementation. different ones are appropriate)
<Regenaxer> The Consumer Goods Forum is a non-profit organization
<Regenaxer> they want to propagate it
<Regenaxer> that's their business
<Regenaxer> creating such standards and agreements
<beneroth> how to they get their money? memberships or also pepople paying for those standards?
<beneroth> second was the reason that the wlan standard was not better audited
<Regenaxer> Only membership if I understood correctly
<Regenaxer> I see :)
mtsd has quit [Remote host closed the connection]
roundsf has joined #picolisp
<tankfeeder> Regenaxer:
alexshendi has quit [Read error: Connection reset by peer]
<tankfeeder> at the same time i can hide number inside "word" without idx
<tankfeeder> i miss something
<tankfeeder> i dont understand something
<Regenaxer> hmm, "word" needs to be quoted
<Regenaxer> I meant this abstract
<Regenaxer> (let W (parseSomething) (if (idx 'B W T) (inc W) (set W 1)))
<tankfeeder> the same
<Regenaxer> In your example
<Regenaxer> (set "word" 1)
<tankfeeder> fighting against in hour
<tankfeeder> this work
<Regenaxer> youd need (set '"word" 1)
<Regenaxer> or (setq ...
<Regenaxer> hehe :)
<Regenaxer> ah
<Regenaxer> one more
<Regenaxer> the above is too simple
<Regenaxer> if the word is in the index, you must use *that* word
<Regenaxer> (if (idx 'B W T) (inc (car @)) (set W 1))
<tankfeeder> make sense
<Regenaxer> 'idx' returns the cell
<tankfeeder> trying again
<Regenaxer> yeah
<Regenaxer> Transient symbols are always tricky, as they look the same
<tankfeeder> old version 1.1secs, your version 0.7secs
<tankfeeder> :)
<tankfeeder> works now
<Regenaxer> wow, great!
<tankfeeder> current version
<Regenaxer> Seems perfect :)
<tankfeeder> i gonna commit then
<Regenaxer> yes, fine
<beneroth> cool stuff
<beneroth> tankfeeder, you did benchmark with python?
<tankfeeder> just on eye
<tankfeeder> speed the same :)
<beneroth> oh okay
<beneroth> nice
<beneroth> haha, about the same amount of code as python 3.6, but no imports
<beneroth> ah, apparently this rosettacode task was demoted to draft exactly because no exact definition for word was initially given
alexshendi has joined #picolisp
<beneroth> Regenaxer, why is (mis> '+Entity) called "mis" ?
<beneroth> for "mismatch" or what?
<beneroth> or does it come from another used term/tool ?
<Regenaxer> yes, mismatch or misfit or so ;)
<Regenaxer> or German "Mist"
<beneroth> haha, nice
<beneroth> mieserabler datensatz
<Regenaxer> yep!
<Regenaxer> It needed a negative term, as NIL means OK
<beneroth> ok, so it does not go back to another term or tool I just don't know
<beneroth> I see
<Regenaxer> right
<beneroth> in swiss german, "mis" is "meins"
<beneroth> ah
<Regenaxer> oh, did not know
<beneroth> Regenaxer, and (mis>) is, to be exact, usually (only?) implemented on the relationship, not on the entity object per se?
<beneroth> like, the relationships themselves are fine but the combination is bad...
<Regenaxer> Could also be in an overridden method
<beneroth> (arguably some relationships are bad then)
<Regenaxer> yes
<Regenaxer> but can be anything
<beneroth> ok
<Regenaxer> also checking other values in the object
<beneroth> thx
coffeecup12345 has quit [Ping timeout: 258 seconds]
rudha has joined #picolisp
alexshendi has quit [Read error: Connection reset by peer]
alexshendi has joined #picolisp
<Regenaxer> beneroth: rudha sends me an updated version of the PDF. The other one was from the unofficial edit mode, that's why the black bubbles
<Regenaxer> beneroth, New (draft!) version:
<beneroth> Regenaxer, ok! so it looked like!
<Regenaxer> :)
<Regenaxer> Questions to rudha :)
<beneroth> will review :)
<Regenaxer> thanks :)
<beneroth> but probably not until thursday or later, I'm in the middle of two deadlines.. last minute changes etc :)
<beneroth> who made the layout and illustrations? looks pretty good (for enterprise audience)
<beneroth> (picolisps prefer text or raw html ^^)
<Regenaxer> so hurry
<Regenaxer> It is estudio, a catalan design company
<beneroth> cool
<beneroth> I might need some graphic designers eventually
<beneroth> catalan is escalating, eh?
<Regenaxer> they are very good
<Regenaxer> yes :(
<beneroth> I read some background stuff that the whole independence movement came up (to the recent events) as a result of years of stalled talks with Spain's central government
<beneroth> but that side seems to obviously get lost in the mainstream media
<Regenaxer> T
alexshendi has quit [Quit: Yaaic - Yet another Android IRC client -]
alexshendi has joined #picolisp
<beneroth> Regenaxer, would be (tell *Pid ..) legal (sending tell to Pid which happens to be the own Pid) ? Assuming the current process is a valid child (not the root process)
<Regenaxer> I expect this will not send anything, as the parent sends to all except the original sender
<Regenaxer> but not sure if such a pathological case is handled
<Regenaxer> yes, checked. I think it will not send anything
<beneroth> ok
<beneroth> use case: I have a worker process which takes function calls from other processes. Currently it is impossible that such a called function (executing within the worker process) would generate such a worker process call itself, but so I know I have to actively prevent this
<Regenaxer> hmm, confusing
<Regenaxer> I would rather expect that you are on the safe side
<Regenaxer> ie. not "have to actively prevent this"
<Regenaxer> As the 'tell' never comes back to the sender
<beneroth> well with this I mean "do never use (tell *WorkerPid ...) in functions which are executed (directly or indirectly via other functions) within the worker process
<beneroth> oh right
<beneroth> the worker process doesn't have the *WorkerPid globale anyway
<beneroth> so this would end up in error anyway... haha :)
<Regenaxer> ok
<beneroth> multiple function executions stalled by concurrent (dbSync) is no problem, right?
<Regenaxer> deadlock?
<beneroth> e.g. (simplified): (tell *Worker 'dbSync) (tell *Worker 'dbSync) (tell *Worker 'dbSync)
<beneroth> no deadlock
<Regenaxer> this should be fine
<beneroth> deadlock is not even possible with your DB locking mechanic
<Regenaxer> T
<beneroth> as a locking process never has to wait for read access
<beneroth> dead lock is only possible in schemes with granular locking
<Regenaxer> and only one gets write access
<beneroth> yeah I guess such waiting executions (similar to co-routines, right?) are just using a bit stack memory (stack right?) ?
<beneroth> (hm..probably completely different implemented than co-routines... I mean from programmer user view it is like multiple function calls running in parallel when they run async due (wait))
<Regenaxer> yes, coroutines don't wait usually
<Regenaxer> just 'yield'
<beneroth> ok
<Regenaxer> deadlocks occur if two processes wait mutually for thing locked by the other
<beneroth> async running function executions are not running in parallel, only one is running at the same time, but one cannot tell which one or in which order as this depends on the (wait)s those executions have. about this way?
<Regenaxer> What do you mean with "async running"?
<beneroth> @deadlocks: yeah, and in your DB locking mechanic a process is always either waiting or locking, but never both
<Regenaxer> yes, an for the same issue
<beneroth> (tell *Worker 'foo) (tell *Worker 'bar) -> depending on calls to e.g. (dbSync) the order of execution of 'foo and 'bar might be much different from the order of the (tell)s
<Regenaxer> yes, right
<Regenaxer> not synchronous at all
<beneroth> but they get executed serially?
<Regenaxer> even without dbSync
<Regenaxer> no
<Regenaxer> the kernel schedules the processel
<beneroth> it is all the same process
<Regenaxer> right, *Worker
<Regenaxer> ok
<beneroth> so within the process, the function gets executed which has all its resources it (wait)s for available during kernel scheduled process time
<beneroth> which is db, but also file descriptors
<Regenaxer> yes, but in this case one after the other
<Regenaxer> first 'foo'
<beneroth> yes, not truly parallel like co-routines
<Regenaxer> As 'tell' goes via pipes
<beneroth> T
<beneroth> but as foo might do (dbSync) while 'bar does (out "someFile"), 'foo might enter (wait) mode and 'bar gets executed until it happens to (wait), yeah?
<Regenaxer> no, it will not get out of dbSync
<Regenaxer> it sets *Run to NIL iirc
<beneroth> I had such a situation and both functions wrote logs, so from the logs it looked quite like parallel execution because the order of start/finish was wildly mixed
<beneroth> I think
<beneroth> both functions competed for (dbSync) and (I guess) write access to the log file
rudha has quit [Quit: Leaving]
<beneroth> where is my mental model wrong?
<Regenaxer> hmm, I think that foo will wait
<beneroth> it is a bit longer ago, I would have to check again
<beneroth> oh I just missed rudha
<Regenaxer> : (pp 'dbSync) (de dbSync (Obj) (let *Run NIL (while (lock (or Obj *DB)) (wait 40) ) (sync) ) )
<Regenaxer> So this blocks
<Regenaxer> He wrote to you in private chat he said
<beneroth> yes he did
<beneroth> I just saw now when he left
<Regenaxer> But retired now
<beneroth> yeah
<Regenaxer> No hurry
<beneroth> I haven't to say anything yet :)
<Regenaxer> I posted the draft pdf link above
<beneroth> yep got it
<Regenaxer> yeah
<Regenaxer> also no hurry
<beneroth> yep
<beneroth> "While the result of the execution of prg is NIL, a select system call is executed for all file descriptors and timers in the VAL of the global variable *Run"
<beneroth> so (dbSync) is shadowing *Run, but what is about the select on the fd's ?
<beneroth> hm
* beneroth is not sure if he is derailing or getting to the goal
<Regenaxer> yes, this is complicated
<beneroth> Regenaxer, we don't have to discuss it now. not important. I don't want to occupy your evening if you have other things to attend to
<Regenaxer> the *Run machinery handles actually also the outer handlers iirc
<beneroth> ok
<Regenaxer> not handles, but shadows them
<Regenaxer> the higher-level value of *Run is saved
<Regenaxer> push (EnvTask) # <L IV> Save task list
<Regenaxer> ...
<Regenaxer> ld Y (Run) # Get '*Run'
<Regenaxer> ld (L I) Y # Save it
<Regenaxer> ld (EnvTask) Y
<Regenaxer> ...
<beneroth> well the reference sounds like both *Run entries and select for all fd's are checked/executed during (wait)
<Regenaxer> yes
<beneroth> so what gets shadowed by whom then?
<Regenaxer> all in the *curren* run list
<beneroth> yeah of course
<Regenaxer> in the C version
<Regenaxer> if (!memq(car(x), taskSave)) {
<Regenaxer> so it handles only those that are not already in select() in the outer list
<Regenaxer> This does not apply to dbSync
<Regenaxer> as it sets *Run to NIL anyway
<beneroth> as I understand it the recent discussion was about the *Run mechanism in general
<Regenaxer> yes, in general
<Regenaxer> but we started with dbSync
<beneroth> during (dbSync) *Run is shadowed to NIL, so within the (wait) only the fd's are checked, correct?
<Regenaxer> only the pipes to the children
<Regenaxer> and hear and mic
<beneroth> and when we happen to listen to a socket or such?
<Regenaxer> Speaker, not mic
<Regenaxer> it is not select()ed
<beneroth> or are those also installed in *Run ?
<Regenaxer> as it is in *Run
<beneroth> ah okay
<beneroth> ok, so the only thing not in *Run is the pil IPC stuff?
<Regenaxer> everything is in *Run, only the child/parent pipes are not
<Regenaxer> yep
<Regenaxer> hard-coded
<beneroth> ok, so the " a select system call is executed for all file descriptors and ..." in the reference to (wait) actually means "for all file descriptors related to pil IPC (there are no others handled outside of *Run)" ?
* beneroth is NOT suggesting to change ref
<Regenaxer> yes, this pil ipc is implicit. the programmer does not know about it :)
<beneroth> T
<Regenaxer> it is documented for 'tell' et al
<Regenaxer> So right, the ref is fine this way I think
<beneroth> why don't incoming (upd) calls (from (commit 'upd) in other processes) getting executed during the (wait) within (dbSync) ? or do they?
<Regenaxer> (upd) is a normal function call
<Regenaxer> arrives via ipc
<beneroth> @reference: yeah even the (tell) ref has an implicit "don't tell to yourself, weirdo" in "it should be the PID of such a process,"
<Regenaxer> they must be executed, very important
<beneroth> even nice rfc-compliant use of "should".. as nothing bad happens if you don't obey it, just nothing happens
<beneroth> T
<beneroth> but so.. having (upd) doing anything else than (wipe) might be..quite dangerous ?
<beneroth> e.g. it may not access DB
<beneroth> s/may/must
<Regenaxer> yes, this process would hang then
<Regenaxer> probably
<beneroth> both of this comments are about the (upd) thing, right? :D
<beneroth> (or could the (tell *Pid) result in hang?)
<Regenaxer> no, not the tell
<Regenaxer> this is asynchronous
<beneroth> ok
<beneroth> just wanted to be sure I don't mix something up :)
<Regenaxer> even if pipe buffer size is exceeded
<beneroth> alright
<Regenaxer> yeah
<Regenaxer> (upd) would hang
<beneroth> yeah of course
<beneroth> why did you even create upd and not use (commit 'wipe) ? so it could be overwritten with some special custom caching mechanism? or so that it is shorter? :P
<beneroth> or did you have use cases where you redefined 'upd ?
<Regenaxer> it is different
<Regenaxer> 'wipe' is evaluating
<Regenaxer> so it would need lots of quotes
<beneroth> ah
<Regenaxer> and space is precious here
<beneroth> hahaha
<beneroth> nice one
<beneroth> you got me!
<Regenaxer> as the pipe space is expensive
<beneroth> haha, this is an advanced picolisp riddle
<Regenaxer> this is also a reason for a short name "upd" :)
<Regenaxer> yeah !
<beneroth> well I currently do (tell *Worker 'event-process 'oneExternalSymbol)
mickiebyrd has quit [Quit: Connection closed for inactivity]
<beneroth> ah
<beneroth> one more question
<beneroth> Regenaxer, del is the destructive version of delete, and (therefore) faster than delete, right?
<Regenaxer> no, del is not destructive in the real sense
<beneroth> ok
<beneroth> I wondered because the word "destructive" was missing
<Regenaxer> setting a symbol value is not considered destructive
<Regenaxer> ok
<beneroth> so what is the destructive function to delete a element in a list?
<Regenaxer> I think it is only with (and (member ..) (con @ (cddr @))) or so
<beneroth> ok, so manual destruction
<Regenaxer> yep
<Regenaxer> I don't see any built-in for that
<beneroth> ok then I saw right
<Regenaxer> So 'del' is not for speed but convenience
<beneroth> T
<Regenaxer> needs 'var' only once
<beneroth> yeah I want to use delete
<Regenaxer> ok
<beneroth> would destructive operation be quite a speed-up here? (I wonder why there isn't a destructive variant)
alexshendi has quit [Quit: Yaaic - Yet another Android IRC client -]
<Regenaxer> usually they are not really faster
<Regenaxer> it is more for the side-effects
<beneroth> ok
<beneroth> well skipped memory allocation
<Regenaxer> well, 'flip' is a lot faster than 'reverse'
<beneroth> yeah
<Regenaxer> and 'sort' would be very expensive
<Regenaxer> if non-desc
<Regenaxer> non-dest
<beneroth> and for delete operation, it only has to keep the last element in a pointer so it can slice it together with the next cell if the current one is the element to be removed, no?
* beneroth is just theorizing
<Regenaxer> as I suggested with member+con above?
<beneroth> yeah. there member is used as seek-operation and @ as the pointer to the last element
<Regenaxer> yep
<beneroth> well nevermind. so you never had the need for it.
<Regenaxer> a typical case it to limit the length of a queue
<Regenaxer> eg in lib/form.l:
<Regenaxer> (and (nth "*Lst" 99) (con @))
<beneroth> I'm too lazy to do any benching about this now or soonish :P just asking "why" and "why not" :)
<Regenaxer> The list is limited to 99
<beneroth> oh, haha
<beneroth> nice one
<beneroth> just slice off the end
<beneroth> s/end/tail
<beneroth> ok
<beneroth> so there is no destructive variant of del because it is not needed often, and when it is needed it can easily be achieved manually
<Regenaxer> yes, thats my feeling
* beneroth grokked it
<beneroth> as always, thank you for your time and patience!
<beneroth> much appreciated
<Regenaxer> it is interesting for me too
<beneroth> good
<beneroth> :)
<Regenaxer> otherwise I forget the details
<beneroth> btw. I happen to use (iter) more and more often. I still have some stuff which works with full lists out of laziness, but (iter) is really nice once you grokked it
<Regenaxer> yes, and most efficient
<beneroth> yeah
<beneroth> database foreach
<Regenaxer> a bit tedious, to build the keys manually
<Regenaxer> In the beginning there was only 'init' and 'step'
<beneroth> only you have to apply it to something different than a +Key
<Regenaxer> then 'iter' and 'scan'
<Regenaxer> +Key is direct, but +Ref etc. neet cons pairs
<beneroth> yeah I also used (scan) a bit.. to prevent loading of the object when I have all the information in the (anyway used) indexes
<Regenaxer> T
<Regenaxer> 'iter' and 'scan' is truely asynchronou
<beneroth> (iter) also skips objects with the wrong class I assume? (when iterating over inherited index relation)
<Regenaxer> no
<Regenaxer> 'init' and 'step' I mean :)
<beneroth> ah
<beneroth> yes
<Regenaxer> 'iter' needs to be exited when you want to stop
<Regenaxer> and to be asynchronous you need a coroutine with them
<Regenaxer> 'step' keeps its own stack
<beneroth> well (iter) is the DB variant of (mapc)
<Regenaxer> correct
<Regenaxer> or (seq) perhaps
<Regenaxer> like mapc I mean
<beneroth> hm, no
<Regenaxer> it maps a whole db file
<beneroth> (seq) and (step) are similar, and similar to (cdr) on lists
<beneroth> it only goes 1 step
<Regenaxer> true
<Regenaxer> seq makes sense only in a loop usually
<beneroth> yes. though you might want to start at different starting nodes.
<Regenaxer> I never did 'seq' on a single arbitrary object
<beneroth> I have a little helper GUI which shows all nodes in a db files with seq. and how it matches the block size in %, e.g. 100% = equal to block size
<Regenaxer> cool
<beneroth> just for fun and to transparently inspect the db file and db structure
<Regenaxer> yes, good for optimization
<Regenaxer> size %
<beneroth> good for understanding. for optimization you want to do this and get a median of the blocksize usage etc
<beneroth> yeah
<beneroth> you work with average or with median?
<Regenaxer> I do sometimes (mapcar size (collect ... to check
<beneroth> yeah exactly
<beneroth> same
<Regenaxer> Hmm, what was the difference of average and median?
<Regenaxer> average is clear
<Regenaxer> (/ (sum ... Lst) (length Lst))
<Regenaxer> (/ (sum size Lst) (length Lst))
<beneroth> (de median (Lst)
<beneroth> (if (bit? 1 N)
<beneroth> (let N (length Lst)
<beneroth> (get (sort Lst) (/ (inc N) 2))
<beneroth> (setq Lst (nth (sort Lst) (/ N 2)))
<beneroth> (/ (+ (car Lst) (cadr Lst)) 2) ) ) )
<beneroth> (I hope this implementation is correct. didn't check for a long time)
<beneroth> median does weight outliers less than average does
<Regenaxer> I see, largest and smallest
<Regenaxer> instead of (bit? 1 N) you could use */ ?
<Regenaxer> cause rounding
<beneroth> oh this code happens to be from you? at least it is written so in my comment xD
<Regenaxer> (*/ N 2)
<beneroth> ah, right
<beneroth> T
<beneroth> ah maybe I took it from rosetta
<Regenaxer> hmm, I don't remember
<Regenaxer> yes, found it
<Regenaxer> seems I wrote it ;)
<Regenaxer> yep
<beneroth> well back then everything on rosetta was from you, afaik
<Regenaxer> have my original version here
<Regenaxer> T
<beneroth> now it could be tankfeeder :)
<Regenaxer> yeah!
<beneroth> so yeah, I used this median to find the optimal block sizes
<Regenaxer> cool
<beneroth> I guess you usually keep all objects of the same class in the same db files (over the lifetime of an application)?
<Regenaxer> yes
<Regenaxer> but if necessary I would change
<Regenaxer> just keeping in mind that 'seq' does not deliver all objects any more
<beneroth> I consider having (dbs) auto-generated in my extra db layer, which might include switching of db file for better match of block usage. but I think I would try to move the existing objects then too, else it just becomes messy.
<Regenaxer> yes, but moving is tricky
<Regenaxer> all references must be fixed
<beneroth> Regenaxer, what do you mean about 'seq? I think it does, given that the DB does not change during the 'seq-stepping?
<Regenaxer> dbMap I thnk
<beneroth> Regenaxer, yep. exactly
<beneroth> dbMap? where is that?
<Regenaxer> you must seq two files then
<Regenaxer> @lib/too.l
<beneroth> (of course)
<Regenaxer> :)
<beneroth> oh wow
<Regenaxer> I used dbMap to migrate old single-file dbs to multi-file
<Regenaxer> In the beginning (before 2005) pil had only a single file in the db
<Regenaxer> and a block size of 64
<Regenaxer> so it got slow for many objects and many indexes
<beneroth> wow. much happened in the recent years
<beneroth> ok
<beneroth> I started programming in 2005.
<Regenaxer> good :)
<beneroth> so what does dbMap take care of and what not?
<Regenaxer> hmm, it can do anything
<Regenaxer> see also @lib/db32-64.l
<beneroth> so it takes care of all +Link's for such a object which changes its name?
<beneroth> by going through all objects in the database?
<Regenaxer> I don't remember the details
<Regenaxer> yes
<beneroth> ok
<beneroth> I have to study that stuff in detail
<beneroth> I might be able to do some of the operations easier, as I store additional metadata in my extra layer.
<beneroth> of course your implementation is more solid, as it doesn't require any metadata to be correct .)
<beneroth> :)
<Regenaxer> yes, dbMap might be useful for that
<beneroth> I probably want to do such operations eventually while the DB is up and running
<beneroth> but a clean maintenance rebuild is nothing to ignore :)
<beneroth> your tooling is extremely good
<Regenaxer> thanks!