Coding: Fleeting Thoughts

A place to discuss the implementation and style of computer programs.

Moderators: phlip, Moderators General, Prelates

User avatar
headprogrammingczar
Posts: 3072
Joined: Mon Oct 22, 2007 5:28 pm UTC
Location: Beaming you up

Re: Coding: Fleeting Thoughts

Postby headprogrammingczar » Fri Jan 31, 2014 10:09 pm UTC

Well, hear it one more time. The stuff edwardk describes in that reddit post are not just things that you can do in Haskell but not Scala, but quite easy things in Haskell besides.

I recommend if you want to really get ahead with Haskell, the best thing to do is combine real-world APIs with the easier parts of the type system. There's a few problems that currently only have unusable C FFI interfaces and equally unusable experimental highly advanced interfaces. A few places, particularly graphics and UI, are still missing that middle ground where you save time in both learning and using.

Also, join freenode/#haskell sometime. We're good for everything from beginner questions to code review to help with higher-level design problems.
<quintopia> You're not crazy. you're the goddamn headprogrammingspock!
<Weeks> You're the goddamn headprogrammingspock!
<Cheese> I love you

User avatar
Jplus
Posts: 1721
Joined: Wed Apr 21, 2010 12:29 pm UTC
Location: Netherlands

Re: Coding: Fleeting Thoughts

Postby Jplus » Sat Feb 01, 2014 12:31 am UTC

ahammel wrote:Hey, look at that, I haven't posted since November.

So, I've inherited a code base of some 16KSLOC of Python and have been asked to impose unit tests on it. This is probably going to mean rewriting large chunks of it.

On the upside, it's modular enough that for some of the rewrites we're not restricted to Python if we don't feel it's the tool for the job. I've been reading up on what we might want to use. It would be really nice to have:

1. Good concurrency support (we've got a Beowulf cluster to ourselves to run this on)
2. Good FP support
3. A static type system
4. Decent runtime

Any thoughts? I'm thinking Scala might be worth looking into.

I agree with everyone who encourages you to use Haskell. C++ might be second choice, it ticks all the marks except for number 1. There are lots of concurrency libraries available for C++ however, so you might be able to find something that suits your needs.

The operating system is at least as important for parallel programming on a cluster. You probably want to use MPI and/or single system image clustering. I know that Boost has MPI bindings for C++, and I suppose there exist bindings for Haskell as well. I see that the MPI wikipage also mentions OCaml and CLR, so those might be interesting options as well.
"There are only two hard problems in computer science: cache coherence, naming things, and off-by-one errors." (Phil Karlton and Leon Bambrick)

coding and xkcd combined

(Julian/Julian's)

User avatar
Thesh
Made to Fuck Dinosaurs
Posts: 6579
Joined: Tue Jan 12, 2010 1:55 am UTC
Location: Colorado

Re: Coding: Fleeting Thoughts

Postby Thesh » Sat Feb 01, 2014 12:58 am UTC

Code: Select all

catch (Exception e) { Logger.Log(e); }

*sigh*
Summum ius, summa iniuria.

User avatar
ahammel
My Little Cabbage
Posts: 2135
Joined: Mon Jan 30, 2012 12:46 am UTC
Location: Vancouver BC
Contact:

Re: Coding: Fleeting Thoughts

Postby ahammel » Sat Feb 01, 2014 1:03 am UTC

Jplus wrote:I agree with everyone who encourages you to use Haskell. C++ might be second choice, it ticks all the marks except for number 1.


C++ has good functional programming support?

C and C++ have Python interop going for them, so that's worth bearing in mind as well. On the other hand, Haskell is awesome and I would be totally psyched if I managed to convince somebody to let me write it for a living.

Jplus wrote:The operating system is at least as important for parallel programming on a cluster. You probably want to use MPI and/or single system image clustering.

Administration of the cluster is pretty well out of my hands, but it is an SSI-type setup and MPI is supported.

Jplus wrote:I know that Boost has MPI bindings for C++, and I suppose there exist bindings for Haskell as well. I see that the MPI wikipage also mentions OCaml and CLR, so those might be interesting options as well.


Hadn't considered OCaml, I'll look into that. We're mostly a Linux shop, which makes me hesitant to think about CLR. I'm not sure how painful it is to develop under Mono.
He/Him/His/Alex
God damn these electric sex pants!

EvanED
Posts: 4331
Joined: Mon Aug 07, 2006 6:28 am UTC
Location: Madison, WI
Contact:

Re: Coding: Fleeting Thoughts

Postby EvanED » Sat Feb 01, 2014 1:09 am UTC

ahammel wrote:
Jplus wrote:I agree with everyone who encourages you to use Haskell. C++ might be second choice, it ticks all the marks except for number 1.


C++ has good functional programming support?
I would say that C++ has... somewhat rudimentary functional programming support, but it does have some. It's better than many languages, but I definitely wouldn't call it good. (I consider true closures to be a prerequisite for that label.) It depends what you're after.

User avatar
Jplus
Posts: 1721
Joined: Wed Apr 21, 2010 12:29 pm UTC
Location: Netherlands

Re: Coding: Fleeting Thoughts

Postby Jplus » Sat Feb 01, 2014 9:46 am UTC

EvanED wrote:
ahammel wrote:
Jplus wrote:I agree with everyone who encourages you to use Haskell. C++ might be second choice, it ticks all the marks except for number 1.


C++ has good functional programming support?
I would say that C++ has... somewhat rudimentary functional programming support, but it does have some. It's better than many languages, but I definitely wouldn't call it good. (I consider true closures to be a prerequisite for that label.) It depends what you're after.

Right... I thought FP meant "floating point". :P

Functional programming support in C++11 is quite extensive, I think you can even mimic nearly all functional constructs from Haskell, but it's not exactly straightforward to use. With something like Boost.Phoenix functional programming in C++ might be even kind of pleasant, though.

As for "true closures", what's not true about C++11 lambdas?
"There are only two hard problems in computer science: cache coherence, naming things, and off-by-one errors." (Phil Karlton and Leon Bambrick)

coding and xkcd combined

(Julian/Julian's)

korona
Posts: 495
Joined: Sun Jul 04, 2010 8:40 pm UTC

Re: Coding: Fleeting Thoughts

Postby korona » Sat Feb 01, 2014 12:25 pm UTC

EvanED wrote:
ahammel wrote:
Jplus wrote:I agree with everyone who encourages you to use Haskell. C++ might be second choice, it ticks all the marks except for number 1.


C++ has good functional programming support?
I would say that C++ has... somewhat rudimentary functional programming support, but it does have some. It's better than many languages, but I definitely wouldn't call it good. (I consider true closures to be a prerequisite for that label.) It depends what you're after.

C++11 lambdas are true closures.

While I think that Haskell is a very good language I don't know if I'd recommend it for commercial projects. Haskell is not that widely used and there are not that much Haskell programmers as there are C++ or Python programmers. The same applies to libraries.

Ubik
Posts: 1016
Joined: Thu Oct 18, 2007 3:43 pm UTC

Re: Coding: Fleeting Thoughts

Postby Ubik » Sat Feb 01, 2014 1:19 pm UTC

A mostly C/C++ dev on Linux question: is there some preferred way to have a per-user directory tree for self-compiled libraries and their include files?

As far as I know, usually package managers install libraries and headers under /usr. When a library is self-compiled, the default seems to be to put the files under /usr/local when make install or equivalent is run.

I would like to be able to only use this particular set of libs and headers only when I'm working on my own projects, to prevent potential breakage of other programs and to have a place I can clean up more confidently, knowing that removing or otherwise messing up with things in there will be breaking my own programming projects at most.

So far I have just put the needed files into a lib directory under the project and handled the include paths and linkage manually, but it doesn't feel very clean approach, and makes use of pkgconfig and similar helpers (whatever was the name of the cmake thing with similar purpose...) inconvenient too.

User avatar
ahammel
My Little Cabbage
Posts: 2135
Joined: Mon Jan 30, 2012 12:46 am UTC
Location: Vancouver BC
Contact:

Re: Coding: Fleeting Thoughts

Postby ahammel » Sat Feb 01, 2014 7:06 pm UTC

korona wrote:While I think that Haskell is a very good language I don't know if I'd recommend it for commercial projects. Haskell is not that widely used and there are not that much Haskell programmers as there are C++ or Python programmers. The same applies to libraries.
It's not a commercial project, as it happens, but that's definitely something I'm keeping in mind. I don't think it should be a game-breaker by itself, though, just because it would be a shame to keep using unsuitable tools just because that's what everybody else uses.

I think my plan will be to hack together some library code that shows Haskell (or C++) to best advantage for our problems, demo it for the management, and see what happens when the time comes to start rewriting.
He/Him/His/Alex
God damn these electric sex pants!

EvanED
Posts: 4331
Joined: Mon Aug 07, 2006 6:28 am UTC
Location: Madison, WI
Contact:

Re: Coding: Fleeting Thoughts

Postby EvanED » Sat Feb 01, 2014 7:48 pm UTC

korona wrote:C++11 lambdas are true closures.
C++11 lambdas don't properly capture their environment, and hence I don't consider them true closures.

To be more specific: C++11 lambdas give you two options for capturing variables in the containing scope, but neither one gives you a true closure. If you capture by reference (int x; ... [&x](int y) { return x + y; } or whatever the syntax is) you don't have a true closure because you can't return the lambda from the function that creates it and still have a usable object. If you capture by copy (int; ...[x](int y) { x += y; } or whatever) then you don't have a true closure because updates to the variable don't affect the surrounding scope. There is no mechanism for achieving both.

I will admit that this is more of a theoretical concern than a practical one because you typically only need one or the other and in C++11 you can pick the appropriate version for your needs. But that still doesn't make them true closures by my definition, and the C++ approach is very occasionally insufficient.

Ubik wrote:A mostly C/C++ dev on Linux question: is there some preferred way to have a per-user directory tree for self-compiled libraries and their include files?

As far as I know, usually package managers install libraries and headers under /usr. When a library is self-compiled, the default seems to be to put the files under /usr/local when make install or equivalent is run.
You can pass --prefix to configure for autocraptools projects to get it to install to something like ~/local, and then set the PKG_CONFIG_PATH environment variable to point to ~/local/lib/pkgconfig.

You might also check out the GNU stow program, which makes dealing with package installations without a package manager a bit more sane.

User avatar
Yakk
Poster with most posts but no title.
Posts: 11128
Joined: Sat Jan 27, 2007 7:27 pm UTC
Location: E pur si muove

Re: Coding: Fleeting Thoughts

Postby Yakk » Sat Feb 01, 2014 8:50 pm UTC

EvanED wrote:
korona wrote:C++11 lambdas are true closures.
C++11 lambdas don't properly capture their environment, and hence I don't consider them true closures.

To be more specific: C++11 lambdas give you two options for capturing variables in the containing scope, but neither one gives you a true closure. If you capture by reference (int x; ... [&x](int y) { return x + y; } or whatever the syntax is) you don't have a true closure because you can't return the lambda from the function that creates it and still have a usable object. If you capture by copy (int; ...[x](int y) { x += y; } or whatever) then you don't have a true closure because updates to the variable don't affect the surrounding scope. There is no mechanism for achieving both.

I will admit that this is more of a theoretical concern than a practical one because you typically only need one or the other and in C++11 you can pick the appropriate version for your needs. But that still doesn't make them true closures by my definition, and the C++ approach is very occasionally insufficient.
So, your problem is the lack of mandatory garbage collection of automatic storage variables.

Stick everything into a smart pointer (to whatever degree of smartness you require -- full gc pointers are pain to use in C++, but shared_ptr is usually more than good enough, with manual cycle breaking) and capture everything by value. That is what garbage collected languages do under the hood anyhow.
One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision - BR

Last edited by JHVH on Fri Oct 23, 4004 BCE 6:17 pm, edited 6 times in total.

korona
Posts: 495
Joined: Sun Jul 04, 2010 8:40 pm UTC

Re: Coding: Fleeting Thoughts

Postby korona » Sat Feb 01, 2014 9:23 pm UTC

Yes, if you want that you can just capture shared_ptrs by value. Of course, if you want maximal performance it is more sane to avoid the overhead of shared_ptr and use a dedicated object for that purpose.

Yakk wrote:[...]full gc pointers are pain to use in C++[...]

Are there any full-tracing-gc-smart-pointer implementations for C++? I don't think that this is possible without a great amount of hacks / support built into the compiler.

EvanED
Posts: 4331
Joined: Mon Aug 07, 2006 6:28 am UTC
Location: Madison, WI
Contact:

Re: Coding: Fleeting Thoughts

Postby EvanED » Sat Feb 01, 2014 9:36 pm UTC

Yakk wrote:So, your problem is the lack of mandatory garbage collection of automatic storage variables.
That's one way of phrasing it. Another is that C++ lacks any automatic mechanism for promoting a stack variable to the heap when it is used in a lambda in a way that requires it.

Stick everything into a smart pointer (to whatever degree of smartness you require -- full gc pointers are pain to use in C++, but shared_ptr is usually more than good enough, with manual cycle breaking) and capture everything by value. That is what garbage collected languages do under the hood anyhow.
I'm not saying you can't get the effect of closures in C++. But if "you can get the effect of closures" is all a languages needs to have "support for closures", then any language supports closures. :-)

Finally, I should say that my closure argument as well as some other reasons I wouldn't say C++ has "good FP support" isn't so much a value judgement of C++. In the context of that language, the limitations of C++11 lambdas are quite natural. My point is just that I don't think you get to then turn around and pretend the limitations aren't there in the face of other languages that do really have true closures and other things that make FP support much better than in C++. (Maybe "fairly rudimentary" in my earlier post was too harsh, and something like "middling" would have been better.)

User avatar
Yakk
Poster with most posts but no title.
Posts: 11128
Joined: Sat Jan 27, 2007 7:27 pm UTC
Location: E pur si muove

Re: Coding: Fleeting Thoughts

Postby Yakk » Sat Feb 01, 2014 10:42 pm UTC

EvanED wrote:
Yakk wrote:So, your problem is the lack of mandatory garbage collection of automatic storage variables.
That's one way of phrasing it. Another is that C++ lacks any automatic mechanism for promoting a stack variable to the heap when it is used in a lambda in a way that requires it.

C++ mandates no stack. Many implementations use a stack.

It has automatic storage duration variables which are destroyed and leave scope when the block they are in exits. A few other languages have this feature as well, but many make variables last as long as there is anyone who "captures" them in every case. That feature is crippling if you want to be able to use RAII style techniques.

Creating data that last longer than the current scope requires a different syntax in C++ than "int x;". You need to place the o invoke the free store. The easiest way is to do a "auto x = std::make_shared<Type>( construction arguments );" in C++11.

Finally, if your code is functional, then your code has no side effects, and everything you capture is both unchanged and unchangable after being captured. :p
I'm not saying you can't get the effect of closures in C++. But if "you can get the effect of closures" is all a languages needs to have "support for closures", then any language supports closures. :-)
No, I'm saying that automatic storage variables may not be what you want in every case.

Automatic storage variables are destroyed, and nobody is allowed to refer to them after they are destroyed. The fact that your capture of them doesn't extend their lifetime is just a matter of not really believing the language when it says "destroyed".
One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision - BR

Last edited by JHVH on Fri Oct 23, 4004 BCE 6:17 pm, edited 6 times in total.

EvanED
Posts: 4331
Joined: Mon Aug 07, 2006 6:28 am UTC
Location: Madison, WI
Contact:

Re: Coding: Fleeting Thoughts

Postby EvanED » Sat Feb 01, 2014 11:21 pm UTC

Yakk wrote:
EvanED wrote:
Yakk wrote:So, your problem is the lack of mandatory garbage collection of automatic storage variables.
That's one way of phrasing it. Another is that C++ lacks any automatic mechanism for promoting a stack variable to the heap when it is used in a lambda in a way that requires it.

C++ mandates no stack. Many implementations use a stack.
You're quibbling. The standard doesn't mandate a hardware stack in the common sense (e.g. you could put stack frames in the heap), but the behavior of automatic variables can only be implemented in something which behaves enough like a stack that the label fits fine. But fine, if you really want, you could pretend I said "C++ lacks any automatic mechanism for promoting an automatic variable to the heap."

A few other languages have this feature as well, but many make variables last as long as there is anyone who "captures" them in every case. That feature is crippling if you want to be able to use RAII style techniques.
I can at least sort of buy that (not sure I completely do, I'd have to think about it more)... but it's getting into value judgements in terms of having true closures vs having better RAII. Saying "implementing true closures means hampering RAII" doesn't refute my statement that "C++ doesn't have true closures." :-)

Finally, if your code is functional, then your code has no side effects, and everything you capture is both unchanged and unchangable after being captured. :p
I'm undecided on whether "functional programming" implies "pure functional". My answer at the moment is "no", because that is convenient for my argument. :-)

Automatic storage variables are destroyed, and nobody is allowed to refer to them after they are destroyed. The fact that your capture of them doesn't extend their lifetime is just a matter of not really believing the language when it says "destroyed".
And the fact that automatic variables that have been captured in a "closure" can be destroyed means that the closure isn't a true closure.

User avatar
Yakk
Poster with most posts but no title.
Posts: 11128
Joined: Sat Jan 27, 2007 7:27 pm UTC
Location: E pur si muove

Re: Coding: Fleeting Thoughts

Postby Yakk » Sat Feb 01, 2014 11:44 pm UTC

"C++ lacks any method to prevent automatic variables from being destroyed at the end of their scope." Because that is the fundamental property of "automatic" variables, they are automatically destroyed for you.

Destruction is just mutating the data that leaves further access of the data as undefined. (What, you want a language without undefined behavior? The strong correctness guarantee on everything? Pfah)

The end of a scope implies a mutating operation on automatic storage variables that destroys them. Think of it as an automatic injected "destroy(x)" at the end of x's scope.

You can, as I mentioned, create data that is *not* automatically destroyed in C++, and you can capture it. But just because C++ injects strong destroys on certain types of variables doesn't mean they aren't closures. They are closures who captured data that was mutated into an invalid state by the program as the programmer instructed!

Imagine if the automatic destruction could be blocked by capture. Now you capture a pointer by reference. Some other code deletes what the pointer points to, quite explicitly. You dereference the elsewhere deleted pointer it in your closure -- do you not expect undefined behavior? Or do you expect the closure to duplicate the entire program state? If the second, how is that not a pure functional argument, and capture by value should always be done?

(You could imagine that every automatic storage variable was allocated on the heap, then code to delete it from the heap was injected at the end of each scope, and the named variable is just an alias set up to refer to the data on the heap. No stack is used.)

If your capture is impure, then C++ has fine captures: it just mutates some captured data at the end of certain scopes automatically on you (to avoid that, don't capture automatic storage variables -- the ones that you asked to be auto destroyed by making them automatic storage -- by reference). If your capture is pure, then C++ has fine captures (it just does not enforce purity on you).

Now, what I'd be curious about is if we can pull off closures in the C++ template metaprogramming language.
One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision - BR

Last edited by JHVH on Fri Oct 23, 4004 BCE 6:17 pm, edited 6 times in total.

User avatar
chridd
Has a vermicelli title
Posts: 843
Joined: Tue Aug 19, 2008 10:07 am UTC
Location: ...Earth, I guess?
Contact:

Re: Coding: Fleeting Thoughts

Postby chridd » Sun Feb 02, 2014 12:50 am UTC

EvanED wrote:To be more specific: C++11 lambdas give you two options for capturing variables in the containing scope, but neither one gives you a true closure. If you capture by reference (int x; ... [&x](int y) { return x + y; } or whatever the syntax is) you don't have a true closure because you can't return the lambda from the function that creates it and still have a usable object. If you capture by copy (int; ...[x](int y) { x += y; } or whatever) then you don't have a true closure because updates to the variable don't affect the surrounding scope. There is no mechanism for achieving both.
If being able to have updates to variables affect the surrounding scope is a requirement for closures, then wouldn't that mean that pure functional languages can't have closures? (Is there even a standard definition for what closures should do when they capture mutable variables?)
~ chri d. d. /tʃɹɪ.di.di/ (Phonotactics, schmphonotactics) · she · Forum game scores
mittfh wrote:I wish this post was very quotable...

User avatar
Negrebskoh
Posts: 139
Joined: Fri Mar 01, 2013 11:49 pm UTC
Location: The Netherlands

Re: Coding: Fleeting Thoughts

Postby Negrebskoh » Sun Feb 02, 2014 11:09 am UTC

chridd wrote:
EvanED wrote:To be more specific: C++11 lambdas give you two options for capturing variables in the containing scope, but neither one gives you a true closure. If you capture by reference (int x; ... [&x](int y) { return x + y; } or whatever the syntax is) you don't have a true closure because you can't return the lambda from the function that creates it and still have a usable object. If you capture by copy (int; ...[x](int y) { x += y; } or whatever) then you don't have a true closure because updates to the variable don't affect the surrounding scope. There is no mechanism for achieving both.
If being able to have updates to variables affect the surrounding scope is a requirement for closures, then wouldn't that mean that pure functional languages can't have closures? (Is there even a standard definition for what closures should do when they capture mutable variables?)

Or all of them, if you take the position that the set of 'variables to affect' is always empty.

Ubik
Posts: 1016
Joined: Thu Oct 18, 2007 3:43 pm UTC

Re: Coding: Fleeting Thoughts

Postby Ubik » Sun Feb 02, 2014 8:40 pm UTC

EvanED: Thanks for reminding me of stow. I read about it recently, but forgot about it soon afterwards. It's not completely necessary when using --prefix and a bunch of environment variables, but should help keeping the development libraries a bit tidier.

I think I'll make some script like "devel-mode [command]" that lets me run a command with those environment variables set or without parameters just starts bash with the changed environment. Should be good enough for most stuff.

Edit: stow, not stew!
Last edited by Ubik on Mon Feb 03, 2014 1:35 pm UTC, edited 1 time in total.

User avatar
phlip
Restorer of Worlds
Posts: 7572
Joined: Sat Sep 23, 2006 3:56 am UTC
Location: Australia
Contact:

Re: Coding: Fleeting Thoughts

Postby phlip » Mon Feb 03, 2014 2:34 am UTC

Am I a terrible person?

Code: Select all

decorator = functools.partial(functools.partial, functools.partial)

@decorator
def foo(func, *args):
  print "foo(%s, %s)" % (func, args)
  return func(*args)

@foo
def bar(x,y):
  print "bar(%s, %s)" % (x, y)

bar(1, 2)
# prints:
# foo(<function bar at 0x15ec1b8>, (1, 2))
# bar(1, 2)

[edit] It's extensible:

Code: Select all

curry = lambda n: functools.partial(*(n * [functools.partial]))

@curry(3)
def foo(a,b,c):
  print "foo(%s,%s,%s)" % (a,b,c)

foo(1)(2)(3) # prints: foo(1,2,3)

Code: Select all

enum ಠ_ಠ {°□°╰=1, °Д°╰, ಠ益ಠ╰};
void ┻━┻︵​╰(ಠ_ಠ ⚠) {exit((int)⚠);}
[he/him/his]

User avatar
Negrebskoh
Posts: 139
Joined: Fri Mar 01, 2013 11:49 pm UTC
Location: The Netherlands

Re: Coding: Fleeting Thoughts

Postby Negrebskoh » Mon Feb 03, 2014 8:01 am UTC

phlip wrote:Am I a terrible person?

Code: Select all

decorator = functools.partial(functools.partial, functools.partial)

@decorator
def foo(func, *args):
  print "foo(%s, %s)" % (func, args)
  return func(*args)

@foo
def bar(x,y):
  print "bar(%s, %s)" % (x, y)

bar(1, 2)
# prints:
# foo(<function bar at 0x15ec1b8>, (1, 2))
# bar(1, 2)

*twitch*

Ubik
Posts: 1016
Joined: Thu Oct 18, 2007 3:43 pm UTC

Re: Coding: Fleeting Thoughts

Postby Ubik » Mon Feb 03, 2014 10:34 am UTC

That took embarrasingly long for me to get. This is how I semi-expanded it if I'm not the only one:

Code: Select all

decorator = functools.partial(functools.partial, functools.partial)

foo = decorator(foo) = functools.partial(functools.partial, foo)

bar = foo(bar) = functools.partial(foo, bar)

bar(*args) = foo(bar, *args)

User avatar
Xeio
Friends, Faidites, Countrymen
Posts: 5101
Joined: Wed Jul 25, 2007 11:12 am UTC
Location: C:\Users\Xeio\
Contact:

Re: Coding: Fleeting Thoughts

Postby Xeio » Mon Feb 03, 2014 3:37 pm UTC

I think I have to go read up on python decorators...

User avatar
PM 2Ring
Posts: 3713
Joined: Mon Jan 26, 2009 3:19 pm UTC
Location: Sydney, Australia

Re: Coding: Fleeting Thoughts

Postby PM 2Ring » Tue Feb 04, 2014 12:46 pm UTC

Xeio wrote:I think I have to go read up on python decorators...

Decorators are one of the most awesomest parts of Python, IMHO.

I've used decorators a little bit, but that was a while ago, so I'm a bit rusty. But here's an example from code I wrote a year or so ago.

Code: Select all

#Euclid's GCD algorithm, modified to return first two remainders < sqrt(a)
def euc(a, b):
    n = a
    while a*a > n:
        a, b = b, a%b
    return a, b

#Simple memoization, with initialised cache
def initialise(cache={}):   
    def memoize(f):
        def memf(x):
            if x not in cache:
                cache[x] = f(x)
            return cache[x]
        return memf
    return memoize

#Find the pair of integers a, b whose squares sum to prime p, p%4 != 3
#memoization doesn't actually save much time...
@initialise({1:(1, 0), 2:(1, 1)})
def sqpair(p):
    #Find sqrt(-1) mod p
    r = imodp(p)
    #Find a,b: a*a + b*b = p
    return euc(p, r)

#Find sqrt(-1) mod p, p%4 == 1
def imodp(p):
    #if p == 2: return 1
    q = p - 1
    j = -1
    while q:
        q, r = divmod(q, 2)
        if r:
            break
        j += 1
    q = 2*q + 1
   
    ok = False
    for a in potential_primes():
        r = pow(a, q, p)
        for i in xrange(j):
            rr = (r*r) % p
            if rr == p - 1:
                ok = True
                break
            elif rr == 1:
                break
            r = rr
        if ok:
            break

    #Use smaller root
    if r * 2 > p:
        r = p - r
    return r

This tutorial looks pretty good... I think I better read it myself. Also check out the docs for the contextlib module.

User avatar
Xeio
Friends, Faidites, Countrymen
Posts: 5101
Joined: Wed Jul 25, 2007 11:12 am UTC
Location: C:\Users\Xeio\
Contact:

Re: Coding: Fleeting Thoughts

Postby Xeio » Tue Feb 04, 2014 6:45 pm UTC

PM 2Ring wrote:This tutorial looks pretty good... I think I better read it myself. Also check out the docs for the contextlib module.
That was actually the one I read. Though I think the complicated part is just the functools.partia, at least that's what took me a while to "get".

Unrelatedly, I wish C# had Ruby's dictionary 'default' value.

Spoiler:

Code: Select all

static void SetValue(Dictionary<string, Dictionary<string, Dictionary<string, Dictionary<string, int>>>> dict, string slot, string level, string glyph, string glyphCount, int value)
{
    Dictionary<string, Dictionary<string, Dictionary<string, int>>> dict1;
    if (!dict.TryGetValue(slot, out dict1))
    {
        dict1 = new Dictionary<string, Dictionary<string, Dictionary<string, int>>>();
        dict[slot] = dict1;
    }
    Dictionary<string, Dictionary<string, int>> dict2;
    if (!dict1.TryGetValue(level, out dict2))
    {
        dict2 = new Dictionary<string, Dictionary<string, int>>();
        dict1[level] = dict2;
    }
    Dictionary<string, int> dict3;
    if (!dict2.TryGetValue(glyph, out dict3))
    {
        dict3 = new Dictionary<string, int>();
        dict2[glyph] = dict3;
    }
    dict3[glyphCount] = value;
}


Somewhat relatedly to that, I can't for the life of me figure out how to get typescript modules to work... so I just set the "compile everything to one .js file" and am not using modules...

User avatar
phlip
Restorer of Worlds
Posts: 7572
Joined: Sat Sep 23, 2006 3:56 am UTC
Location: Australia
Contact:

Re: Coding: Fleeting Thoughts

Postby phlip » Wed Feb 05, 2014 12:04 am UTC

Xeio wrote:Though I think the complicated part is just the functools.partia, at least that's what took me a while to "get".

Well, where it came from is that I think the usual boilerplate for a wrapper-style decorator is a bit tedious:

Code: Select all

def foo(func):
  @functools.wraps(func)
  def wrapper(*args, **kwargs):
    code that eventually calls:
    return func(*args, **kwargs)
  return wrapper

@foo
def bar(...):
  ...
Ugly, and I often forget the "return wrapper" line at the end. And so, inspired by contextlib.contextmanager, I wanted to make a decorator that could turn a much simpler form of function into the form that is needed to make a decorator. I thought, how can I make it look like this instead:

Code: Select all

def foo(func, *args, **kwargs):
  code that eventually calls:
  return func(*args, **kwargs)

def bar(...):
  ...
bar = funcutils.partial(foo, bar)
except, that's not a decorator... to make it a decorator we need to partial again:

Code: Select all

@funcutils.partial(funcutils.partial, foo)
def bar(...):
  ...
but it would instead be neat if we could use "@foo" as normal... so we need to reassign:

Code: Select all

foo = funcutils.partial(funcutils.partial, foo)
and if we want to do that as a decorator, we need to use partial again:

Code: Select all

@funcutils.partial(funcutils.partial, funcutils.partial)

Of course, if I was actually considering doing this in real code, I'd just write:

Code: Select all

def decorator(func):
  @functools.wraps(func)
  def wrapper(f):
    @functools.wraps(f)
    def innerwrapper(*args, **kwargs):
      return func(f, *args, **kwargs)
    return innerwrapper
  return wrapper

Code: Select all

enum ಠ_ಠ {°□°╰=1, °Д°╰, ಠ益ಠ╰};
void ┻━┻︵​╰(ಠ_ಠ ⚠) {exit((int)⚠);}
[he/him/his]

Ciber
Posts: 126
Joined: Fri Mar 15, 2013 1:33 pm UTC

Re: Coding: Fleeting Thoughts

Postby Ciber » Wed Feb 05, 2014 4:15 am UTC

I found this kinda cool sounding programming language called Hoon at http://www.urbit.org/
It strikes me that it would be really cool to make a science forum game with an entire world coded in Hoon. Players would have to hack the work through a very limited sort of eval function. I am thinking giving players a stat like "mana" which determines the number of characters then can use. Hoon already has this fancy shmancy table of magic words to use, and every default function can be called using them.
Main characters: Almost all functions in Hoon are called with a combo of two of these, which is spoken my just adding the two subwords together.
Spoiler:

Code: Select all

ace  space      gal  <          per  )
bar  |          gar  >          sel  [
bas  \          hax  #          sem  ;
buc  $          hep  -          ser  ]
cab  _          kel  {          sig  ~
cen  %          ker  }          soq  '
col  :          ket  ^          tar  *
com  ,          lus  +          tec  `
doq  "          pam  &          tis  =
dot  .          pat  @          wut  ?
fas  /          pel  (          zap  !

Irregular combinations:
Spoiler:

Code: Select all

--    hephep    phep   
+-    lushep    slep
++    luslus    slus
==    tistis    stet
+<    lusgal    glus
+>    lusgar    gras
-<    hepgal    gelp
->    hepgar    garp 

I remember seeing a renaming for numbers too, but I cannot find it now.

User avatar
ahammel
My Little Cabbage
Posts: 2135
Joined: Mon Jan 30, 2012 12:46 am UTC
Location: Vancouver BC
Contact:

Re: Coding: Fleeting Thoughts

Postby ahammel » Wed Feb 05, 2014 6:21 am UTC

Ciber wrote:I found this kinda cool sounding programming language called Hoon at http://www.urbit.org/

On the one hand, I am pretty intrigued by this Urbit thing, but on the other hand the devs are clearly from Mars.


"We built a really small virtual machine and something about air travel and the Internet has failed!"
He/Him/His/Alex
God damn these electric sex pants!

Ciber
Posts: 126
Joined: Fri Mar 15, 2013 1:33 pm UTC

Re: Coding: Fleeting Thoughts

Postby Ciber » Wed Feb 05, 2014 12:35 pm UTC

Well yeah, the site in general gives off the same kind of vibe as that one "The earth is actually flat" site.

User avatar
Steax
SecondTalon's Goon Squad
Posts: 3038
Joined: Sat Jan 12, 2008 12:18 pm UTC

Re: Coding: Fleeting Thoughts

Postby Steax » Fri Feb 07, 2014 8:03 am UTC

Vagrant is awesomely practical for my web work. It makes a new VM where I execute it, installs a fresh new install of whatever dependencies I need, then runs a bash script to set everything up. It then links an internal directory with the directory I installed it in, and turns on port forwarding. From there, I can SSH into the VM, freeze/thaw it every evening and morning, and deploy multiple boxes for each of my projects. So each of them run in an isolated environment using only their bare minimum requirements, making testing much easier. And as a bonus, the install script guarantees I have a clean way to deploy my app when I push it on a VPS.

All of the config files (1 vagrantfile, 1 boostrap script) go with version control and live with the rest of my code. I can chuck it at my coworkers and it's dead simple to use. Even the most junior designers know how to type "vagrant up" and browse to localhost:8088 (or something).
In Minecraft, I use the username Rirez.

lgw
Posts: 437
Joined: Mon Apr 12, 2010 10:52 pm UTC

Re: Coding: Fleeting Thoughts

Postby lgw » Fri Feb 07, 2014 9:29 pm UTC

Xeio wrote:Unrelatedly, I wish C# had Ruby's dictionary 'default' value.

Spoiler:

Code: Select all

static void SetValue(Dictionary<string, Dictionary<string, Dictionary<string, Dictionary<string, int>>>> dict, string slot, string level, string glyph, string glyphCount, int value)
{
    Dictionary<string, Dictionary<string, Dictionary<string, int>>> dict1;
    if (!dict.TryGetValue(slot, out dict1))
    {
        dict1 = new Dictionary<string, Dictionary<string, Dictionary<string, int>>>();
        dict[slot] = dict1;
    }
    Dictionary<string, Dictionary<string, int>> dict2;
    if (!dict1.TryGetValue(level, out dict2))
    {
        dict2 = new Dictionary<string, Dictionary<string, int>>();
        dict1[level] = dict2;
    }
    Dictionary<string, int> dict3;
    if (!dict2.TryGetValue(glyph, out dict3))
    {
        dict3 = new Dictionary<string, int>();
        dict2[glyph] = dict3;
    }
    dict3[glyphCount] = value;
}


Somewhat relatedly to that, I can't for the life of me figure out how to get typescript modules to work... so I just set the "compile everything to one .js file" and am not using modules...


Annoyingly, the ConcurrentDictionary class has a good method to simplify this pattern: GetOrAdd(key, default), which adds the key with the default value to the dictionary (and returns it) if there isn't a value already. It's frustrating that normal Dictionary doesn't have this syntax sugar. I get that it's needed for ConcurrentDictionary for atomicity, but wow the pattern you illustrate here gets old!
"In no set of physics laws do you get two cats." - doogly

User avatar
Yakk
Poster with most posts but no title.
Posts: 11128
Joined: Sat Jan 27, 2007 7:27 pm UTC
Location: E pur si muove

Re: Coding: Fleeting Thoughts

Postby Yakk » Fri Feb 07, 2014 10:14 pm UTC

Why wouldn't you just write:

Code: Select all

Y GetOrDefault( Dictionary<X, Y>& dict1, X key, Y(*)() default )

well, in the C# syntax. (Y(*)() is a nullary function returning Y, Dictionary<X,Y>& is a read/write parameter of type Dictionary<X,Y>

Then your code reduces to:

Code: Select all

    Dictionary<string, Dictionary<string, Dictionary<string, int>>> dict1 = GetOrDefault(
      dict, slot,
      ()=>new Dictionary<string, Dictionary<string, Dictionary<string, int>>>()
    );
    Dictionary<string, Dictionary<string, int>> dict2 = GetOrDefault(
      dict1, level,
      ()=>new Dictionary<string, Dictionary<string, int>>()
    );
    Dictionary<string, int> dict3 = GetOrDefault(
      dict2, glyph,
      ()=> new Dictionary<string, int>()
    );
    dict3[glyphCount] = value;

which seems less annoying slightly.

Or write an augmented Dictionary?
One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision - BR

Last edited by JHVH on Fri Oct 23, 4004 BCE 6:17 pm, edited 6 times in total.

User avatar
Xeio
Friends, Faidites, Countrymen
Posts: 5101
Joined: Wed Jul 25, 2007 11:12 am UTC
Location: C:\Users\Xeio\
Contact:

Re: Coding: Fleeting Thoughts

Postby Xeio » Sun Feb 09, 2014 9:39 pm UTC

Yakk wrote:Or write an augmented Dictionary?
Eh, I only really was using this to transform a spreadsheet into a more manageable .json format.

Though a GetOrDefault extension method may have been nice at least in that I could have used var to avoid repeating the type signature. Sometimes copy paste laziness wins in personal projects.

User avatar
phlip
Restorer of Worlds
Posts: 7572
Joined: Sat Sep 23, 2006 3:56 am UTC
Location: Australia
Contact:

Re: Coding: Fleeting Thoughts

Postby phlip » Sun Feb 09, 2014 11:29 pm UTC

Does anyone have any great suggestions for lightweight Python web frameworks? There's a site I'm building, which currently uses raw CGI, which isn't the most efficient thing ever, but it's what the host supports... using the builtin cgi module and some random templating module I found... which works, but I feel like I should be doing things differently. I'm basically using Python to emulate half of any given "reasons PHP sucks" article... code embedded in data, URIs that follow the filesystem, etc.

However, I was already planning to move everything over to a VPS, which will give me more control over everything, and I won't be restricted to just using mod_cgi to run my code... and I've only written 2 rather simple pages so far, so I'm not averse to rewriting the whole thing in a new framework. The site is mostly just read-only views of data from other applications (which is what the VPS is actually for), so it doesn't need to be too fancy, but if there's something suitable out there, I'd love to use it.

I've never written web stuff in Python at all before this, so I have no idea what's available...

Code: Select all

enum ಠ_ಠ {°□°╰=1, °Д°╰, ಠ益ಠ╰};
void ┻━┻︵​╰(ಠ_ಠ ⚠) {exit((int)⚠);}
[he/him/his]

User avatar
Robert'); DROP TABLE *;
Posts: 730
Joined: Mon Sep 08, 2008 6:46 pm UTC
Location: in ur fieldz

Re: Coding: Fleeting Thoughts

Postby Robert'); DROP TABLE *; » Sun Feb 09, 2014 11:35 pm UTC

Although I don't have much experience in Python on the web, Flask is quite usable. It lets you easily avoid the URI=filesystem trap and seperate your content from the code. (And "Hello world" is literally 3 lines of non-boilerplate.)
...And that is how we know the Earth to be banana-shaped.

User avatar
phlip
Restorer of Worlds
Posts: 7572
Joined: Sat Sep 23, 2006 3:56 am UTC
Location: Australia
Contact:

Re: Coding: Fleeting Thoughts

Postby phlip » Sun Feb 09, 2014 11:41 pm UTC

This... actually looks pretty ideal, thanks!

Code: Select all

enum ಠ_ಠ {°□°╰=1, °Д°╰, ಠ益ಠ╰};
void ┻━┻︵​╰(ಠ_ಠ ⚠) {exit((int)⚠);}
[he/him/his]

heatsink
Posts: 86
Joined: Fri Jun 30, 2006 8:58 am UTC

Re: Coding: Fleeting Thoughts

Postby heatsink » Mon Feb 10, 2014 1:00 am UTC

Today I demonstrated that you can crash a C++ program by reinterpreting the type of an array using only "safe" implicit casts. The guy I showed this to was surprised that this seemingly-obvious error generates no warnings with g++ or clang.

The fundamental problem is that pointers have two incompatible meanings. A pointer-to-T may be

  • A pointer to any class that inherits from T (including T itself), or
  • A pointer to an array of T.

Array subscripting (with a nonzero subscript) is only valid with the second kind of pointer, but C++'s type system allows either kind of pointer to be subscripted.

Code: Select all

struct Base {
  virtual void foo(void) {}
};

struct Derived : public Base {
  int i;
};

int main() {
  Base *p = new Derived[10];
  p[3].foo(); // boom
  return 0;
}
Last edited by heatsink on Mon Feb 10, 2014 1:04 am UTC, edited 1 time in total.

User avatar
Aaeriele
Posts: 2127
Joined: Tue Feb 23, 2010 3:30 am UTC
Location: San Francisco, CA

Re: Coding: Fleeting Thoughts

Postby Aaeriele » Mon Feb 10, 2014 1:01 am UTC

phlip wrote:Does anyone have any great suggestions for lightweight Python web frameworks?

As someone who has done a ton of Python web development, I also recommend Flask.
Vaniver wrote:Harvard is a hedge fund that runs the most prestigious dating agency in the world, and incidentally employs famous scientists to do research.

afuzzyduck wrote:ITS MEANT TO BE FLUTTERSHY BUT I JUST SEE AAERIELE! CURSE YOU FORA!

lgw
Posts: 437
Joined: Mon Apr 12, 2010 10:52 pm UTC

Re: Coding: Fleeting Thoughts

Postby lgw » Mon Feb 10, 2014 6:50 pm UTC

heatsink wrote:Today I demonstrated that you can crash a C++ program by reinterpreting the type of an array using only "safe" implicit casts. The guy I showed this to was surprised that this seemingly-obvious error generates no warnings with g++ or clang.

The fundamental problem is that pointers have two incompatible meanings. A pointer-to-T may be

  • A pointer to any class that inherits from T (including T itself), or
  • A pointer to an array of T.

Array subscripting (with a nonzero subscript) is only valid with the second kind of pointer, but C++'s type system allows either kind of pointer to be subscripted.
Spoiler:

Code: Select all

struct Base {
  virtual void foo(void) {}
};

struct Derived : public Base {
  int i;
};

int main() {
  Base *p = new Derived[10];
  p[3].foo(); // boom
  return 0;
}


One more case of "legacy C baggage harmful in C++". If you use vectors in preference to arrays, you don't run into this sort of problem. Similarly, C pointers are rarely a good parameter in C++: better a reference, unless you're transferring ownership for which a shared_ptr (really, the only reason to use a pointer now is if NULL really means something, or when dealing with funky hardware with interesting stuff at fixed real memory addresses).
"In no set of physics laws do you get two cats." - doogly

User avatar
Yakk
Poster with most posts but no title.
Posts: 11128
Joined: Sat Jan 27, 2007 7:27 pm UTC
Location: E pur si muove

Re: Coding: Fleeting Thoughts

Postby Yakk » Mon Feb 10, 2014 7:05 pm UTC

A pointer to T is never a pointer to an array of T.

A pointer to an array of T is a distinct type from a pointer to T.

You can treat a pointer to the first element of an array of T as a pointer to an array of T, but only with care.

new T[] creates an array, and returns a pointer to the first element. Yet another reason to avoid using new. Sadly, `std::unique_ptr<T[10]>` has the same problem.

std::unique_ptr<std::array<T, 10>> foo = make_unique<std::array<T, 10>>() does not have that quirk -- it returns a pointer to the array itself.
One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision - BR

Last edited by JHVH on Fri Oct 23, 4004 BCE 6:17 pm, edited 6 times in total.


Return to “Coding”

Who is online

Users browsing this forum: No registered users and 6 guests