Page 247 of 250

Re: Coding: Fleeting Thoughts

Posted: Thu Feb 15, 2018 1:05 am UTC
Yeah, a dict just connects keys to values. Values can be anything, keys are anything "hashable" (strings, numbers, tuples of those, and some other things).

An array is basically just a dict that only accept integer keys, and only in a certain range - it just connects the numbers 0, 1, 2, etc to values.

You just use different syntax to define/refer to them. Previous posts already went into detail about what sort of dict you'd want to use - probably with keys that are 3-tuples for the coordinates, like `{(0,0,0): 0, (0,0,1): 0, ...}`; I won't repeat the previous posts here. To get/set them you can still use the `a[foo]` syntax, but it'll look like `a[0,0,1] = 2` (set the key (0,0,1) to the value 2), instead of your current drilling down into a multidimensional array like `a[0][0][1] = 2` (look up the value at index 0, then the value at index 0 of the subarray, then the value at index 1 of the sub-subarray, and set it to 2).

Re: Coding: Fleeting Thoughts

Posted: Thu Feb 15, 2018 2:45 am UTC
Assuming that cells are either alive or dead, I'd go with sets. Sets are simple things that hold a bunch of items. For example, a set in which each item is the coordinate of a live cell. So the question "is the cell at (x,y,z) alive or dead?" becomes "does the coordinate (x,y,z) exist in the set of alive cells or not?". And birthing/killing cells becomes a matter of adding/deleting coordinates to/from the set.

The example below also adds some functional programming to the mix leading to only a few (>5) lines of code:

Code: Select all

`# in haskell typology: getNeighbours :: Coordinate -> Set Coordinatedef getNeighbours(coordinate):    (x,y,z) = coordinate # python 3 doesn't allow deconstructing a tuple in the parameter definition, so it needs its own line    return frozenset({(x+dx,y+dy,z+dz) for dx in [-1,0,1] for dy in [-1,0,1] for dz in [-1,0,1] if not dx == dy == dz == 0}) # butifel    # also, if you want a wrapping field or dead boundaries or whatever, this is the place to go# in haskell typology: evolveState :: Set Coordinate -> Set Coordinatedef evolveState(oldAlive):    newAlive = set() # starting with a blank slate (practice may show that copying the old state and adding/deleting some elements performs better)    activeCells = oldAlive.union(*map(getNeighbours, oldAlive)) # such ugly notation to take the union of a couple of sets (also simply assuming that all live cells and their neighbours may change; this may be optimized)    for cell in activeCells:        aliveNeighbours = sum(1 for neighbour in getNeighbours(cell) if neighbour in oldAlive)        if aliveNeighbours == 5 or (cell in oldAlive and 1 < aliveNeighbours < 8): # copied a rule from the blogosphere            newAlive.add(cell) # I would've gone for `newAlive = newAlive.union({cell})` if it weren't both less readable and significantly slower because python won't realize that it can reuse the set instead of making a modified copy    return frozenset(newAlive) # just pretend it was a frozen set all alongdef __main__():    alive = someInitialLiveCells # like `frozenset({(-1, 0, 0), (0, -1, 0), (0, 0, 0), (0, 1, 0), (1, -1, 0), (1, 1, 0)})`    for generation in range(1, 1001):        alive = evolveState(alive)        somehowShowThisState(generation, alive) # like... I dunno, I avoid graphics libraries, and 3D data doesn't translate to text output nicely`

It becomes more readable if you remove the comments, except for the oldAlive.union(*map(getNeighbours, oldAlive)), which is alive ∪ { N | N ∈ getNeighbours(A), A ∈ alive } or "grab all the alive cells and all their surrounding cells".
I have no idea about the performance of this thing, though I expect something of an O(n²) time per iteration because of the way the active cells are decided.

Re: Coding: Fleeting Thoughts

Posted: Tue Mar 27, 2018 12:30 pm UTC
*Spends a few days writing code at work*

Code: Select all

`me@work:project\$ sloccount src...Total Physical Source Lines of Code (SLOC) = 700Development Effort Estimate, Person-Years (Person-Months) = 0.14 (1.65)...Total Estimated Cost to Develop = \$18,578`

So where's my cut of that \$18k?

Re: Coding: Fleeting Thoughts

Posted: Tue Mar 27, 2018 1:09 pm UTC
There's extensive documentation about the calculation and interpretation of those values. I bet you haven't produced a single UML diagram, barely did any testing, and there's zero documentation, so your 700 loc aren't finished yet.

Sloccount is also highly inaccurate in small projects. Writing the first 700 lines for a new project is cheap (In java, you can write 700 loc just by clicking "create new empty project"!). On the other hand, try getting 700 lines of code commited to the linux kernel, see if you can do it in less than 1.65 months.

Then again, \$18k sounds much better, so run with it. Apparently I've been producing 2 man-years worth of code in half a man-year, so I'm going to ask for a 300% raise tomorrow.

Re: Coding: Fleeting Thoughts

Posted: Tue Mar 27, 2018 1:21 pm UTC
Lines of code have negative value. Every line of code added to our code base is a liability that will cost us money every year until the code base goes under.

Negative lines of code, now that is quality stuff.

Re: Coding: Fleeting Thoughts

Posted: Tue Mar 27, 2018 1:58 pm UTC
Tub wrote:There's extensive documentation about the calculation and interpretation of those values. I bet you haven't produced a single UML diagram, barely did any testing, and there's zero documentation, so your 700 loc aren't finished yet.

To be fair, it has had zero testing.
And basically no error handling.
Also ANSI C.

Yakk wrote:Lines of code have negative value. Every line of code added to our code base is a liability that will cost us money every year until the code base goes under.

Negative lines of code, now that is quality stuff.

There's probably more than one git repo at work where my "Lines Added minus Lines Deleted" statistic is negative.

Re: Coding: Fleeting Thoughts

Posted: Wed Apr 04, 2018 4:00 pm UTC
Ciber wrote:Lately I have been working on force directed graph layouts. Currently trying to port (is that the right word here?) my naive implementation to numpy because with the 400 node, 900 link test graph I'm using it takes several seconds per iteration.

Uhm, I may be a bit late to the party here, but are you still working on this? I'm asking because I just came out of the end-of-semester slog and both force directed graph layouts and general spatial data structures are still fresh in my mind.

Basically, the best approaches I know boil down to desperately avoiding O(N²) runtime and bring it down to O(N*log(N)). Also cutting the number of iterations down can help, but that usually grows slower than the cost of each iteration anyway.

Re: Coding: Fleeting Thoughts

Posted: Fri Apr 13, 2018 7:23 pm UTC
I'm between stuff to do at work, so I've been reading papers about Haskell and implementing the ideas as haskell:y as possible in Java.

Works... surprisingly well once you find a middle road between Java's clunkiness and Haskell's weirdness. But now I have a expressive powerful monadic parser library cooking based on Hutton & Meijer's paper. Took some head scratching before I figured out how to express chainl1 and chainr1, but in the end the were pretty clean too.

Re: Coding: Fleeting Thoughts

Posted: Fri Apr 13, 2018 10:35 pm UTC
I was browsing a computing magazine in the supermarket, the other day (wondering whether to buy it for a couple the whatever-you-call-clickbait-when-it's-on-a-magazine-cover items) and one thing that (subsequent to the original interest) caught my eye was a suggestion to try Ada, as a strong-typed overloadable language that people might prefer to Haskell.

I have to say, Ada was the most tortuous of the strongly-typed languages that I've 'learnt'. It looks like the IDE surrounding the suggested Ada compiler does solve a lot of the issues I had with the (probably) emacs-editing I did, back in the day, but I still have bad memories of its awkwardness, compared to my experience around the same time with Forth, LISP, Fortran and even COBOL. Heck, I'd even prefer to write my missile-flight controller modules1 in Redcode, if I could make it sufficiently Imp-proof!

Anyway, just thought I'd open the vent, slightly, on this long internal simmering disquiet. It quite possibly was the reason I was driven to particularly like Perl, and I'll leave it up to you whether this was ultimately a very good consequence (for me and/or the world at large) or a very bad one (ditto). The Ada of 2018 may even be worth a new try, I suppose, although I've probably forgotten more about it than I ever learnt in the first place (as the post-'80s implemention doubtless was revised and contemperaneously restyled several times to address problems such as I might have identified for myself) so I might effectively be starting from scratch again.

1 The alleged historic reasoning behind its belligerent awkwardness to use. Though I personally think someone didn't like female programmers, and so wanted to blacken Countess Lovelace's name, rather than insult Grace Hopper.

Re: Coding: Fleeting Thoughts

Posted: Sun Jul 29, 2018 10:37 am UTC
Dungeon siege 1

[t:template,n:spell_sky_turret_lightning]
{
category_name = "magic";
doc = "spell sky turret lightning";
specializes = base_spell_good;
[aspect]
{
gold_value = 206;
}
[attack]
{
damage_max = 0;
damage_min = 0;
}
[common]
{
description = "Expels a split of lightning at the Target.";
screen_name = "Turret Lightning";
}
[gui]
{
active_icon = b_gui_ig_i_ic_sp_038;
inventory_icon = b_gui_ig_i_ic_sp_038_inv;
}
[magic]
{
attack_damage_modifier_max = ((#magic+1)*3.17+5)*(1+((1/(#magic+8))+.041));
attack_damage_modifier_min = ((#magic+1)*3.17+5)*(1-((1/(#magic+8))+.041));
cast_range = 54;
effect_duration = 500000;
mana_cost = 1;
mana_cost_modifier = (#magic*0)+1;
max_level = 27000;
required_level = 90;
requires_line_of_sight = false;
speed_bias = 1;
target_type_flags = tt_conscious_enemy | tt_unconscious_enemy | tt_breakable;
usage_context_flags = uc_offensive;
cast_sub_animation = 0;
}
[spell_turret]
{
initial_delay = 0.01;
lightning_dur = 0.01;
shot_rate = 0.01;
effect_script = gom_turret_lightning;
charge_effect = gom_turret_lightning_charge;
}
}

Not even remotely fun, but fun to think about.

Re: Coding: Fleeting Thoughts

Posted: Tue Jul 31, 2018 11:43 am UTC
Has anyone found a valid use case for a WeakMap in JavaScript?

It's supposed to prevent memory leaks, by allowing you to attach data to objects without leaking the data. The example for a key is often a DOM element, because those tend to disappear every now and then.

Code: Select all

`let wm = new WeakMap();let e = document.getElementById('foo');wm.set(e, {  private_data: 1})`

There. The private data can be reclaimed by the GC when the element disappears, so there's no memory leak.

But there wasn't any memory leak in the legacy variant, either:

Code: Select all

`const KEY = '_private_data';let e = document.getElementById('foo');e[KEY] = {  private_data: 1}`

No leak, though the additional property may mess up for...in loops or something; it's not properly hidden. But another ES6 feature does fix that:

Code: Select all

`const KEY = Symbol('private data');`

So.. why would I use a WeakMap to associate data with an object when I can use a Symbol?

Considering that a WeakMap has no size and is not iterable, I don't see any other use than associating data to an object. It's intentionally impossible to use a weakmap to detect when an object was garbage collected.

Re: Coding: Fleeting Thoughts

Posted: Tue Jul 31, 2018 4:30 pm UTC
Note that you *can* retrieve all the Symbols attached to an object; they're not "hidden" in any way whatsoever, they're just guaranteed to never collide with anything else unless you specifically use the same Symbol object. So if you really do have some private data to associate, you can't attach it with a Symbol.

That said, *most* of the time using a Symbol or a WeakMap is just a choice of how you want to interact with the data; whether it makes more sense to think of it as a property of the object or as data collected into a map.

Re: Coding: Fleeting Thoughts

Posted: Tue Jul 31, 2018 5:39 pm UTC
I'm aware, but being collision-free and non-enumerable is enough to prevent accidental bugs when multiple separate modules mess with the same object.

You can access or change private members in Java with the reflection API, but we still consider them private. You can mangle symbol properties in JavaScript using getOwnPropertySymbols(), but unless someone hands you the symbol as part of an API contract, you have no idea what the associated data means, and you know you shouldn't access it. That's private enough for me.

I suppose WeakMap has its uses if you use Object.freeze() a lot, or if you want to delete associated data from all objects (just delete the WeakMap). It's just that I have seen zero use cases where it would prevent memory leaks as advertised. It certainly won't help me prevent the memory leaks I'm facing right now.

Re: Coding: Fleeting Thoughts

Posted: Tue Jul 31, 2018 8:57 pm UTC
I've done some benchmarks.

Code: Select all

`const LOOPS = 10000000;const ITERATIONS = 10;(function() {   const PROP = '_private_data';   let o = {};   bench('prop-single',      function() {         o[PROP] = {            a: 1,            b: 1         };      },      function() {         let p = o[PROP];         let sum = p.a + p.b;         p.a = p.b;         p.b = sum;      }   );   console.log(o[PROP].a);})();(function() {   const PROP_A = '_private_data_a';   const PROP_B = '_private_data_b';   let o = {};   bench('prop-multi',      function() {         o[PROP_A] = 1;         o[PROP_B] = 1;      },      function() {         let sum = o[PROP_A] + o[PROP_B];         o[PROP_A] = o[PROP_B];         o[PROP_B] = sum;      }   );   console.log(o[PROP_A]);})();(function() {   const PROP = Symbol();   let o = {};   bench('symbol-single',      function() {         o[PROP] = {            a: 1,            b: 1         };      },      function() {         let p = o[PROP];         let sum = p.a + p.b;         p.a = p.b;         p.b = sum;      }   );   console.log(o[PROP].a);})();(function() {   const PROP_A = Symbol();   const PROP_B = Symbol();   let o = {};   bench('symbol-multi',      function() {         o[PROP_A] = 1;         o[PROP_B] = 1;      },      function() {         let sum = o[PROP_A] + o[PROP_B];         o[PROP_A] = o[PROP_B];         o[PROP_B] = sum;      }   );   console.log(o[PROP_A]);})();(function() {   let wm = new WeakMap();   let o = {};   bench('weakmap-single',      function() {         wm.set(o, {            a: 1,            b: 1         });      },      function() {         let p = wm.get(o);         let sum = p.a + p.b;         p.a = p.b;         p.b = sum;      }   );   console.log(wm.get(o).a);})();(function() {   let wm_a = new WeakMap();   let wm_b = new WeakMap();   let o = {};   bench('weakmap-multi',      function() {         wm_a.set(o, 1);         wm_b.set(o, 1);      },      function() {         let sum = wm_a.get(o) + wm_b.get(o);         wm_a.set(o, wm_b.get(o));         wm_b.set(o, sum);      }   );   console.log(wm_a.get(o));})();function bench(name, init, func) {   let times = [];   let t = Date.now();   for (let i = 0; i < ITERATIONS; i++) {      init();      for (let l = 0; l < LOOPS; l++) {         func();      }      let t2 = Date.now();      times.push(t2 - t);      t = t2;   }   let avg = times.reduce((a, e) => a + e) / times.length;   console.log(name, times, avg);}`

Code: Select all

`prop-single [ 115, 114, 101, 100, 101, 100, 101, 101, 101, 100 ] 103.4prop-multi [ 332, 309, 290, 293, 292, 294, 292, 293, 284, 293 ] 297.2symbol-single [ 232, 227, 226, 226, 226, 226, 227, 227, 226, 225 ] 226.8symbol-multi [ 253, 239, 241, 240, 264, 251, 241, 240, 240, 248 ] 245.7weakmap-single [ 464, 460, 469, 467, 465, 459, 457, 457, 457, 456 ] 461.1weakmap-multi [ 4180, 4278, 3888, 3791, 3743, 3762, 3700, 3697, 3856, 3701 ] 3859.6`

I'm not sure why the first one is faster, but the WeakMap variants are slowest in all engines I've tested (those numbers are nodejs v10.5)

Re: Coding: Fleeting Thoughts

Posted: Fri Aug 31, 2018 4:38 am UTC
Best Coding Language (or operating system I guess):
ThanOS
"Perfectly balanced, as all things should be."

Re: Coding: Fleeting Thoughts

Posted: Fri Aug 31, 2018 2:34 pm UTC
gd1 wrote:Best Coding Language (or operating system I guess):
ThanOS
"Perfectly balanced, as all things should be."

You'll never run out of hard drive space! As the hard drive gets too full, half of the files are deleted to make room.

Re: Coding: Fleeting Thoughts

Posted: Mon Sep 03, 2018 7:19 am UTC
I don't know what has made me happier - the fact that Scott Hanselman liked and responded to my tweet, or that my boss' boss saw and liked it.

In summary:

Re: Coding: Fleeting Thoughts

Posted: Sun Sep 09, 2018 4:53 pm UTC
I was trying some real world™ Haskell the other day and stumbled on monads within monads, specifically IO [IO a] due to reading from files inside a directory. I can slam it all together as a one-liner \f dir -> listDirectory dir >>= traverse (\file -> fmap f (readFile file)) (or in a slightly pointless style \f -> listDirectory >=> traverse (readFile >=> purify f)), but is there a way to cleanly write this in do notation?

Also, is there a way to tell IO to actually close the current file before going on to the next so it doesn't crash because of "too many files open"? To me it's clear that each element in the returned list is independent from the others, so, when one element is eaten (in my case printed) by the function that maps over the IO[a], it can't affect the next element of the traversal so the file can be closed.
Would a deepSeq help hint GHC that the data has fully been extracted from a file after finishing an IO action?

In the end I just flattened the whole thing with unsafePerformIOs, but now I feel dirty.

Re: Coding: Fleeting Thoughts

Posted: Wed Sep 12, 2018 3:47 pm UTC
Using do notation should just be the obvious "three lines, each pulling the value out of a deeper monad" thing:

Code: Select all

`do files <- listDirectory dir    file <- files    contents <- readFile file    doStuffWithContents contents`

And yeah, to tell IO to close the file you force strictness. Pulling some example code from SO:

Code: Select all

`import Control.Parallel.Strategies (rnf)-- rnf means "reduce to normal form"main = do inFile <- openFile "foo"           contents <- hGetContents inFile          rnf contents `seq` hClose inFile -- force the whole file to be read, then close          putStr contents`

Re: Coding: Fleeting Thoughts

Posted: Sun Sep 16, 2018 5:50 pm UTC
I'm not really sure I understand why this fixes a stack overflow exception I was having. Or why I wasn't having the overflow exception prior to today (was using this on Thursday at least).

https://github.com/Xeio/TwitchPokemonHe ... 47bc01189c

I'm not even entirely sure why I decided to try changing the scope of the variable as a possible solution (ignoring that I should have scoped it to begin with probably). The initialization line shouldn't even run more than once which presumably is the only way the stack overflow could happen (and attaching a debugger seems to prove this to be the case, and yet the variable value is as though the initialization code did run twice).

Re: Coding: Fleeting Thoughts

Posted: Sun Sep 16, 2018 6:35 pm UTC
Xanthir wrote:Using do notation should just be the obvious "three lines, each pulling the value out of a deeper monad" thing:

That's what you'd think at first, yes. But GHC will immediately complain that it can't match [] with IO. And then you realize (>>=) (and related combinators) work with m a -> (a -> m b) -> m b rather than m a -> (a -> othermonad b) -> whichevermonad b.

I just realized you can tactically insert sequence to end up with something like

Code: Select all

`test :: FilePath -> (String -> a) -> IO [a]test dir f = do  files <- listDirectory dir  sequence \$ do    file <- files    pure \$ do      contents <- readFile file      pure \$ f contents`

which still isn't pretty with all the nested dos and dollars, but at least there's no join . fmap sequence involved to smash the result together.

Thanks for the file reader. I really hoped there would be a solution without resorting to handles and seqs, but it can hide as a function :: FilePath -> IO String in some library file anyway.

Re: Coding: Fleeting Thoughts

Posted: Sun Sep 16, 2018 9:30 pm UTC
Flumble wrote:
Xanthir wrote:Using do notation should just be the obvious "three lines, each pulling the value out of a deeper monad" thing:

That's what you'd think at first, yes. But GHC will immediately complain that it can't match [] with IO. And then you realize (>>=) (and related combinators) work with m a -> (a -> m b) -> m b rather than m a -> (a -> othermonad b) -> whichevermonad b.

Oh duh, right, I always forget that do notation uses `bind` and not `fmap` (and there's no way around it...), so you have to care a lot more about your monad stack.

So yeah, it looks like you want to end up with an `IO List a`, when your naive monad stack is an `IO List IO a`. List is Traversable, so throwing a sequence into there will indeed do what you need.

I just realized you can tactically insert sequence to end up with something like

Code: Select all

`test :: FilePath -> (String -> a) -> IO [a]test dir f = do  files <- listDirectory dir  sequence \$ do    file <- files    pure \$ do      contents <- readFile file      pure \$ f contents`

A few notes:

1. Since you're working with these as monads, you can just use the standard `return` rather than `pure`. Makes it a touch more readable; we don't have to think about the Applicative context here.

2. Dont' overuse do when you dont' need to! The innermost one is just calling `f` on the contents of the IO returned by readFile, so you can replace that whole block with `f <\$> readFile file`. (Any time you find yourself writing a two-liner do-block like that, consider just fmapping instead.) This gives:

Code: Select all

`test :: FilePath -> (String -> a) -> IO [a]test dir f = do  files <- listDirectory dir  sequence \$ do    file <- files    pure (f <\$> (readFile file))`

3. Then you end up with another trivial two-liner on the new innermost block, where you're once again just fmap'ing over the structure. You could leave it like this, as it's reasonably readable, but I find it better to rearrange a bit, so you do the sequencing up front. That way you retain a single non-nested `do`, where the RHS of every `<-` is an IO:

Code: Select all

`test :: FilePath -> (String -> a) -> IO [a]test dir f = do  files <- listDirectory dir  openedFiles <- sequence \$ readFile <\$> files  return f <\$> openedFiles`

I think this'll work? I'm writing from my head right now, as I have no interpreter handy.

Edit:

Or, since map + sequence is just traverse, use that directly, duh:

Code: Select all

`test :: FilePath -> (String -> a) -> IO [a]test dir f = do  files <- listDirectory dir  openedFiles <- traverse readFile files  return f <\$> openedFiles`

Edit2:

And at this point we're *nearly* back to your original one-liner: \f dir -> listDirectory dir >>= traverse (\file -> fmap f (readFile file)).

The only significant thing to realize is that the lambda you're passing to traverse is too complicated. If you break down the sequence of events into another map, you recover the structure of my final do block:

Code: Select all

`\f dir -> listDirectory dir >>= traverse readFile >>= return \$ fmap f`

`listDirectory` is `FilePath -> IO [File]`, `traverse readFile` is `[File] -> IO [String]`, and `return \$ fmap f` is `[String] -> IO [a]`, so all the types match up.

It's a little annoying that you have to do the `>>= return` dance at the end, just because you don't care about the IO monad at all with the last chunk of code. Even more annoying is that there's no convenient flipped <\$>, so you can't easily just tack a fmap onto that bind sequence and do away with the extra monadic frippery. But if you create one yourself, you can write it more simply:

Code: Select all

`(<&>) = flip fmap\f dir -> listDirectory dir >>= traverse readFile <&> fmap f`

(I chose <&> because Lens uses & as a flip of \$, so <&> as a flip of <\$> seems to make good sense. Alternatives are something related to |>, the pipeline operator many languages have, with is also a flip of \$; maybe |\$> ?)

Whether this is simpler for you mentally or not may be a personal choice. ^_^

Re: Coding: Fleeting Thoughts

Posted: Sun Sep 16, 2018 9:52 pm UTC
Xeio wrote:
I'm not really sure I understand why this fixes a stack overflow exception I was having. Or why I wasn't having the overflow exception prior to today (was using this on Thursday at least).

https://github.com/Xeio/TwitchPokemonHe ... 47bc01189c

I'm not even entirely sure why I decided to try changing the scope of the variable as a possible solution (ignoring that I should have scoped it to begin with probably). The initialization line shouldn't even run more than once which presumably is the only way the stack overflow could happen (and attaching a debugger seems to prove this to be the case, and yet the variable value is as though the initialization code did run twice).
Phhht, I'm an idiot, I had the extension installed twice, once from the chrome store and once from an unpacked version. Explains some other weird behavior was seeing too...

EDIT: On the other hand, I don't understand why I only saw one toolbar button for the extension so I'm still unsure as to why that would happen.

My guess is it happened because I installed the extension on my laptop to test with, and it probably sync'd to my main computer.

Re: Coding: Fleeting Thoughts

Posted: Mon Sep 24, 2018 10:37 am UTC
How terrible is this?

Code: Select all

`try:   # bunch of method calls on objects from modules   mod_1_object.call_1()   mod_2_object.other_call()   mod_1_object.call_2()   # some stuff   mod_1_object.call_4()   mod_2_object.call_3()except:   e_t, e_v, e_tb = sys.exc_infO()   if any(mod_1.__file__ in trace for trace in traceback.format_tb(e_tb)):      # this will trigger special error handling      raise SomeMagicException(...)   else:      raise`

Errors coming from one module typically indicate a particular failure, usually handled by just waiting for a device controller (represented by mod_1_object) to auto-restart and trying again.
However errors coming from anywhere else cannot be handled.
There isn't a single exception type to handle here - you may get one of several exceptions (some are overly generic) and there is significant overlap between the exceptions coming form mod_1 and mod_2.

I could go into mod_1 and wrap every method there with "try-except-raise", but I don't really like wrapping exceptions without good reason (and Python 2 doesn't have exception chaining).
Or I could surround every relevant call with "try-except" - less hacky but far uglier.

Re: Coding: Fleeting Thoughts

Posted: Mon Sep 24, 2018 6:00 pm UTC
Errors coming from one module typically indicate a particular failure, usually handled by just waiting for a device controller (represented by mod_1_object) to auto-restart and trying again.

"typically"? Is there an exception that cannot be handled by retrying? Exceptions for invalid arguments, maybe? If so, add +1 terrible to your proposal.

I see three possibilities:
a) accept your hack or a variation thereof. I would advise against it unless there's only a single consumer of mod1. If you start copy/pasting that catch block into multiple places you just know how it's going to end.

b) Give mod1 a proper API to signal temporary vs. permanent errors. Unless you like returning EAGAIN, that means subclassing a new DeviceIsAfkPleaseRetryLaterError.

If you worry about losing information about the swallowed exceptions, you can either log them, or jury-rig your own exception chaining. (I'm not fluent in python; in JS I'd just try overriding toString() and getStackTrace() or something). Note that swallowing the exceptions in mod1 isn't any worse than swallowing them in your example code.

c) make mod1's API asynchronous and let it do the retrying on its own. Depending on your actual use case, that may end up being incredibly clean, or incredibly ugly.

Or I could surround every relevant call with "try-except" - less hacky but far uglier.

It avoids using debugging information to guide control flow, but it's verbose. A compromise might be

Code: Select all

`bool can_retry = truetry:   can_retry = true   mod_1_object.call_1()   can_retry = false   mod_2_object.other_call()   can_retry = true   mod_1_object.call_2()   mod_1_object.call_4()   can_retry = false   mod_2_object.call_3()except ArgumentError:  raise // can never be handledexcept:   if (can_retry)      raise SomeMagicException(...)   else:      raise`

Re: Coding: Fleeting Thoughts

Posted: Thu Oct 11, 2018 3:01 pm UTC
Is there any shell out there that supports selecting and copying text on the line? (with support for sensible keybindings like home to go to the start of a line, ctrl+shift+left to select the word to the left)
Most of the time I'm working from within screen, so I can copy anything with ctrl+a,[ but I'd like to know if there's an alternative for the times I'm working directly from a terminal.

Re: Coding: Fleeting Thoughts

Posted: Thu Oct 11, 2018 4:54 pm UTC
A shell can only do very limited selection, because it isn't a terminal. It simply doesn't know (or care) what's on your screen at the moment. A shell can only copy parts of the current command line - and even bash can do that. It'll go into a bash-internal buffer though, so you cannot copy/paste between different sessions.

For complicated selections on your screen content, this will need to be implemented in a terminal. You mentioned screen, which happens to be a terminal emulator, but (unless heavily scripted), its internal paste buffer is per-session, too.

To copy into your system's clipboard, look for features in your actual terminal. There are scripting extensions for both urxvt and konsole that allow mouseless selections. For xterm or PuTTy, you'll need the mouse. Anything else, ask google.

Re: Coding: Fleeting Thoughts

Posted: Thu Oct 11, 2018 7:17 pm UTC
Flumble wrote:Is there any shell out there that supports selecting and copying text on the line? (with support for sensible keybindings like home to go to the start of a line, ctrl+shift+left to select the word to the left)
Most of the time I'm working from within screen, so I can copy anything with ctrl+a,[ but I'd like to know if there's an alternative for the times I'm working directly from a terminal.

Bash has had vi mode forever, which I don't know if you consider more sensible than the default EMACS-like bindings (religious wars is thataway), but it's an option if you like vi(m).

Re: Coding: Fleeting Thoughts

Posted: Thu Oct 11, 2018 10:09 pm UTC
Thanks, I never knew virtually all shells support emacs/vi-style editing (I guess the thought has never occurred because windows shells don't have it). I couldn't find anything with a preliminary search for "shell with selection", obvious in hindsight because the jargon is mark, yank, region, kill ring etc.

Re: Coding: Fleeting Thoughts

Posted: Sat Oct 13, 2018 5:03 am UTC
It's less a bash feature and more a readline feature, so a lot of different CLIs from the GNU space have that sort of functionality.

Re: Coding: Fleeting Thoughts

Posted: Tue Oct 16, 2018 12:10 am UTC
Hmph, I've been messing with mounting an NTFS partition for an hour. Turns out ntfs-3g wasn't installed, while the builtin kernel module reports the mount as "rw" despite being read-only.

Re: Coding: Fleeting Thoughts

Posted: Tue Oct 23, 2018 8:10 pm UTC
IMB MQ's .Net bindings poop all over type safety.

Re: Coding: Fleeting Thoughts

Posted: Tue Nov 13, 2018 3:00 pm UTC
Does anyone know if there's an autoconf macro to find the LIBS, LDFLAGS and CXXFLAGS needed to use C++11 threads? From what I've seen just using -pthread isn't the magic bullet that works for all mainstream compilers, e.g. Clang needs -stdlib=libstdc++ as well and some bugged versions of GCC need -Wl,--no-as-needed.

Re: Coding: Fleeting Thoughts

Posted: Tue Nov 13, 2018 10:20 pm UTC

I asked my friend on the C++ standards committee, and here's his answer:

IIRC (@wakomeup / @LouisDionne can correct me) libstdc++ needs -pthread -lpthread, and libc++ needs nothing special to use std::thread from C++11. A concrete repro is necessary to really get what they're trying to do.

Feel free to respond to him directly for further clarification.

Re: Coding: Fleeting Thoughts

Posted: Wed Nov 14, 2018 7:21 am UTC
So I take it there's no ready-made autoconf macro to figure everything out automatically?

In any case, this helps me along a good way. I can use AC_COMPILE_IFELSE to check for __GLIBCXX__ and add -pthread if found. Then use AC_LINK_IFELSE to see if the threading configuration works. If it doesn't, I guess I'm content to just error out and tell the user to pass whatever flags their (broken) configuration needs. I'll update this once I cobble something together (or fail to do so).

Re: Coding: Fleeting Thoughts

Posted: Wed Nov 14, 2018 9:59 am UTC
clang generally does not require -stdlib=libstdc++. That SO link is quite old, and the runtime error suggests that the then-recent version of libc++ simply lacked support.

I think the autotools way would be to attempt different configurations until one compiles and works (starting with no parameters, then -pthread, then the more exotic variations). Relying on compiler macros is tricky, because compilers (or headers) sometimes lie about these things.

Re: Coding: Fleeting Thoughts

Posted: Wed Nov 14, 2018 1:37 pm UTC
Tub wrote:
I think the autotools way would be to attempt different configurations until one compiles and works (starting with no parameters, then -pthread, then the more exotic variations). Relying on compiler macros is tricky, because compilers (or headers) sometimes lie about these things.

That does seem better. I wrote a small macro to test whether one configuration works:

Code: Select all

`# SYNOPSIS##   AX_CXX_THREAD([EXTRA-CXXFLAGS], [ACTION-IF-TRUE], [ACTION-IF-FALSE])## DESCRIPTION##   Test if C++ standard library threads work, optionally given a set of#   CXXFLAGS.##   This macro attempts to compile and link a piece of stub code using C++11's#   std::thread, after adding EXTRA-CXXFLAGS to CXXFLAGS. If successful, it#   sets##     have_cxx_thread=true##   and runs ACTION-IF-TRUE##   If unsuccessful, it sets##     have_cxx_thread=false##   and runs ACTION-IF-FALSE.##serial 1AC_DEFUN([AX_CXX_THREAD], [    AC_REQUIRE([AC_PROG_CXX])    AC_LANG_PUSH([C++])    oldCXXFLAGS="\${CXXFLAGS}"    CXXFLAGS="\${CXXFLAGS} \$1"    AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include <chrono>#include <thread>]], [[std::thread t([](void){ std::this_thread::sleep_for(            std::chrono::seconds(5)); });]])],        [have_cxx_thread=true],        [have_cxx_thread=false])    AC_LANG_POP([C++])    CXXFLAGS="\${oldCXXFLAGS}"    if test "x\$have_cxx_thread" = "xtrue" ; then        m4_ifvaln([\$2],[\$2],[:])        m4_ifvaln([\$3],[else \$3])    fi])`

Then I just run through some configurations:

Code: Select all

`AC_MSG_CHECKING([if C++11 threads work without extra CXXFLAGS])AX_CXX_THREAD([], [        AC_MSG_RESULT([yes])        AC_SUBST([threading_CXXFLAGS],[""])], [    AC_MSG_RESULT([no])    AC_MSG_CHECKING([if C++11 threads work with -pthread])    AX_CXX_THREAD([-pthread], [        AC_MSG_RESULT([yes])        AC_SUBST([threading_CXXFLAGS], [-pthread])], [    AC_MSG_RESULT([no])    AC_MSG_CHECKING([if C++11 threads work with -pthread -lpthread])    AX_CXX_THREAD([-pthread -lpthread], [        AC_MSG_RESULT([yes])        AC_SUBST([threading_CXXFLAGS], [-pthread -lpthread])], [    AC_MSG_RESULT([no])    AC_MSG_CHECKING([if C++11 threads work with -pthread -stdlib=libstdc++])    AX_CXX_THREAD([-pthread -stdlib=libstdc++], [        AC_MSG_RESULT([yes])        AC_SUBST([threading_CXXFLAGS], [-pthread -stdlib=libstdc++])], [    AC_MSG_RESULT([no])    AC_MSG_CHECKING([if C++11 threads work with -pthread -Wl,--no-as-needed])    AX_CXX_THREAD([-pthread -Wl,--no-as-needed], [        AC_MSG_RESULT([yes])        AC_SUBST([threading_CXXFLAGS], [-pthread -Wl,--no-as-needed])], [    AC_MSG_ERROR([Cannot determine how to use C++11 threads.])])])])])])`

More can be added as needed, of course (-pthread always works with my preferred configuration, and I'm not going out of my way to test vastly different configurations until someone else hits a snag).

Re: Coding: Fleeting Thoughts

Posted: Wed Nov 14, 2018 5:51 pm UTC
Link wrote:More can be added as needed, of course (-pthread always works with my preferred configuration, and I'm not going out of my way to test vastly different configurations until someone else hits a snag).

That seems reasonable.

If you take a closer look at that SO thread you linked, you'll notice that the program compiled just fine, but failed to run. If you wish to support those configurations, you'll actually need to run the program (AC_RUN_IFELSE) instead of just linking, and make the code test for successful thread execution. Something like this:

int exit_code = EXIT_FAILURE;
exit_code = EXIT_SUCCESS;
});
t.join();
return exit_code;

Re: Coding: Fleeting Thoughts

Posted: Wed Nov 14, 2018 6:56 pm UTC
Tub wrote:
Link wrote:More can be added as needed, of course (-pthread always works with my preferred configuration, and I'm not going out of my way to test vastly different configurations until someone else hits a snag).

That seems reasonable.

If you take a closer look at that SO thread you linked, you'll notice that the program compiled just fine, but failed to run. If you wish to support those configurations, you'll actually need to run the program (AC_RUN_IFELSE) instead of just linking, and make the code test for successful thread execution. Something like this:

int exit_code = EXIT_FAILURE;
exit_code = EXIT_SUCCESS;
});
t.join();
return exit_code;

Ha, that's exactly the kind of mistake you make when reading about the bug in the evening and hastily implementing the macro the next day like I did. Will fix tomorrow.

EDIT: the updated macro:

Code: Select all

`# SYNOPSIS##   AX_CXX_THREAD([EXTRA-CXXFLAGS], [ACTION-IF-TRUE], [ACTION-IF-FALSE])#   AX_CXX_THREAD_LINKONLY([EXTRA-CFLAGS], [ACTION-IF-TRUE],#                          [ACTION-IF-FALSE])## DESCRIPTION##   Test if C++ standard library threads work, optionally given a set of#   CXXFLAGS.##   AX_CXX_THREAD attempts to compile, link and run a piece of stub code using#   C++11's std::thread, after adding EXTRA-CXXFLAGS to CXXFLAGS. If#   successful, it sets##     have_cxx_thread=true##   and runs ACTION-IF-TRUE##   If unsuccessful, it sets##     have_cxx_thread=false##   and runs ACTION-IF-FALSE.##   AX_CXX_THREAD_LINKONLY behaves identically, save for the fact that it does#   not attempt to run the compiled and linked program. This is useful when#   cross-compiling.#serial 2AC_DEFUN([AX_CXX_THREAD], [    AC_REQUIRE([AC_PROG_CXX])    AC_LANG_PUSH([C++])    oldCXXFLAGS="\${CXXFLAGS}"    CXXFLAGS="\${CXXFLAGS} \$1"    AC_RUN_IFELSE([AC_LANG_PROGRAM([[#include <thread>]], [[    int exit_code = EXIT_FAILURE;    std::thread t([](int *x){ *x = EXIT_SUCCESS; }, &exit_code);    t.join();    return exit_code;]])],        [have_cxx_thread=true],        [have_cxx_thread=false])    AC_LANG_POP([C++])    CXXFLAGS="\${oldCXXFLAGS}"    if test "x\$have_cxx_thread" = "xtrue" ; then        m4_ifvaln([\$2],[\$2],[:])        m4_ifvaln([\$3],[else \$3])    fi])AC_DEFUN([AX_CXX_THREAD_LINKONLY], [    AC_REQUIRE([AC_PROG_CXX])    AC_LANG_PUSH([C++])    oldCXXFLAGS="\${CXXFLAGS}"    CXXFLAGS="\${CXXFLAGS} \$1"    AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include <thread>]], [[    int exit_code = EXIT_FAILURE;    std::thread t([](int *x){ *x = EXIT_SUCCESS; }, &exit_code);    t.join();    return exit_code;]])],        [have_cxx_thread=true],        [have_cxx_thread=false])    AC_LANG_POP([C++])    CXXFLAGS="\${oldCXXFLAGS}"    if test "x\$have_cxx_thread" = "xtrue" ; then        m4_ifvaln([\$2],[\$2],[:])        m4_ifvaln([\$3],[else \$3])    fi])`
I've opted to explicitly pass a pointer to the return value instead of relying on the lambda's capture-by-reference, but otherwise it's the same. I've defined a second macro AX_CXX_THREAD_LINKONLY that uses AC_LINK_IFELSE, which is useful when cross-compiling.

Re: Coding: Fleeting Thoughts

Posted: Thu Nov 15, 2018 4:16 pm UTC
So there are two projects I want to work on in my copious free time.

1: "Virtual considered harmful":
Play with C++23 reflection and metaclasses/type functions. Write a system that replaces the standard C++ object model with one that permits plain old data polymorphic types, and with vtable split from instance. That permits polymorphic value types, polymorphic views, and polymorphism stored outside of type instance (RLE polymorphism for text, for example). It would also permit runtime polymorphism (dynamic vtables) and other non-C++ OO tricks.

2: "Algebraic immutable git documents":
Take libgit2 and immvec and the like, and build a logically immutable document format that stores its data either serialized in a git blob or deserialized in memory (to permit lazy loading of a document from a file). Figure out ways to cleanly have complex structures (lists, properties, maps) while easily composing "transform object of type foo" and "find object of type foo in document" into "transform document" algebraicly. Leave things open for disk/memory/remote documents to be worked on.