Coding: Fleeting Thoughts

A place to discuss the implementation and style of computer programs.

Moderators: phlip, Moderators General, Prelates

User avatar
Diadem
Posts: 5654
Joined: Wed Jun 11, 2008 11:03 am UTC
Location: The Netherlands

Re: Coding: Fleeting Thoughts

Postby Diadem » Sat Nov 16, 2013 11:37 am UTC

No but I found 22 occurrences of the string literal "3.1415" in the code. About half of them were definitions of some type of pi constant, the rest were direct uses in a calculation. There were 6 different number of digits being used. Granted, the majority of those 22 occurrences were in commented out code, but still. I also found definitions for 2 pi or the 2/pi I mentioned in my previous post. There's also at least one definition of pi as "4 * atan(1)".

I think I do not need to worry about job security for the foreseeable future :)
It's one of those irregular verbs, isn't it? I have an independent mind, you are an eccentric, he is round the twist
- Bernard Woolley in Yes, Prime Minister

korona
Posts: 495
Joined: Sun Jul 04, 2010 8:40 pm UTC

Re: Coding: Fleeting Thoughts

Postby korona » Sat Nov 16, 2013 1:36 pm UTC

I don't see anything wrong in defining pi = 4 * atan(1). IMHO that is more beautiful than specifying a numerical constant directly.

mr-mitch
Posts: 477
Joined: Sun Jul 05, 2009 6:56 pm UTC

Re: Coding: Fleeting Thoughts

Postby mr-mitch » Sat Nov 16, 2013 2:21 pm UTC

korona wrote:I don't see anything wrong in defining pi = 4 * atan(1). IMHO that is more beautiful than specifying a numerical constant directly.


Because you can also do acos(-1)? I think it's more beautiful than the multiplication by 4.

User avatar
Xenomortis
Not actually a special flower.
Posts: 1446
Joined: Thu Oct 11, 2012 8:47 am UTC

Re: Coding: Fleeting Thoughts

Postby Xenomortis » Sat Nov 16, 2013 2:37 pm UTC

Implementation dependent, but works in VC++ and GCC (I think).

Code: Select all

#define _USE_MATH_DEFINES
#include <cmath>
//M_PI for pi


mr-mitch wrote:Because you can also do acos(-1)? I think it's more beautiful than the multiplication by 4.

atan2(0.0, -1.0) == pi
Image

User avatar
Thesh
Made to Fuck Dinosaurs
Posts: 6568
Joined: Tue Jan 12, 2010 1:55 am UTC
Location: Colorado

Re: Coding: Fleeting Thoughts

Postby Thesh » Sat Nov 16, 2013 3:57 pm UTC

Either I have to make this faster (under 30 minutes), or I need to pay for a 32+ core server for about a month or two (which I really can't afford right now).

Code: Select all

uint32_t min_nonlinearity(const uint16_t *sbox) {
    uint32_t i, min=32768;
    for (i=1; i<65536; i++) {
        uint32_t j;
        for (j=1; j<65536; j++) {
            uint32_t nonlinearity = 0, x;
            for (x=0; x<65536;x++) {
                if (__builtin_parity(x&i) == __builtin_parity(sbox[x]&(uint16_t)j)) nonlinearity++;
            }
            if (nonlinearity>32768)
                nonlinearity=65536-nonlinearity;
            if (nonlinearity<min) {   
                min = nonlinearity;
            }
        }
    }   
    return min;
}


I'm doing a bit of research, so I need statistics on various properties of 16x16 bit bijective sboxes constructed through various methods (it's a toy example, as the intended result is to (hopefully) build strong 64x64 bit bijective sboxes using the same method with only 64kB of static storage and very little computational cost).
Summum ius, summa iniuria.

User avatar
skeptical scientist
closed-minded spiritualist
Posts: 6142
Joined: Tue Nov 28, 2006 6:09 am UTC
Location: San Francisco

Re: Coding: Fleeting Thoughts

Postby skeptical scientist » Sat Nov 16, 2013 4:37 pm UTC

I'm assuming __builtin_parity(n) is the number of 1s in the binary representation of n, mod 2?

I'm not sure if this is helpful, but one way to think of it is the following: for each fixed i, you have a collection of 16 vectors in F265536 (the nth vector has given by sbox[x]&(1 << (n-1)), and a target vector given by __builtin_parity(x&i). You want to find the nonzero linear combination of the first 16 vectors which best approximates the target vector. Right now you're doing this by brute force iterating over all 2^16-1 possible nonzero linear combinations (the for loop over j). There should be a better way. I don't remember off the top of my head, but I think algorithms for projecting onto subspaces work for vector spaces over finite fields? I think this will let you cut out one of your for loops, and may even speed up the overall algorithm by a few thousand times.

1) How long does this take now?
2) sbox appears to be a length 65536 array of uint16_t. Where does it come from? Can it be an arbitrary such array? Or do you know anything about it which you may be able to take advantage of in designing an algorithm?
I'm looking forward to the day when the SNES emulator on my computer works by emulating the elementary particles in an actual, physical box with Nintendo stamped on the side.

"With math, all things are possible." —Rebecca Watson

User avatar
ahammel
My Little Cabbage
Posts: 2135
Joined: Mon Jan 30, 2012 12:46 am UTC
Location: Vancouver BC
Contact:

Re: Coding: Fleeting Thoughts

Postby ahammel » Sat Nov 16, 2013 5:17 pm UTC

I know I'm not the first one to complain about this, but I think Python's decision to move `reduce` to `functools` is really silly. If we want to make the point that comprehension expressions are to be preferred to the "higher order function and lambda" pattern, fine, but then why leave `map` and `filter`, which can be trivially replaced by a comprehension, and move `reduce`, which can't?
He/Him/His/Alex
God damn these electric sex pants!

Ubik
Posts: 1016
Joined: Thu Oct 18, 2007 3:43 pm UTC

Re: Coding: Fleeting Thoughts

Postby Ubik » Sat Nov 16, 2013 5:28 pm UTC

Fleeting micro optimization thoughts regarding Thesh's code - I have actually no idea if I'm making sense at all.

If you make the x loop the outermost, you can do the i and j loops without reading the sbox array contents from memory all the time. Well, not memory but cache, but that's still slower than registers, right? Also allows you to move the __builtin_parity(x&i) outside the j loop, though I don't know if it is that slow to begin with.

Thought of this because there was some stackoverflow thread where use of conditionals in the inner loop slowed processing down if the branch predictor was misguessing all the time based on previous iterations. I don't know if that can be helped, but I have a vague memory that at least GCC wasn't smart enough to rearrange the loops. (Intel compiler was.)

User avatar
Xenomortis
Not actually a special flower.
Posts: 1446
Joined: Thu Oct 11, 2012 8:47 am UTC

Re: Coding: Fleeting Thoughts

Postby Xenomortis » Sat Nov 16, 2013 5:30 pm UTC

It takes a long time to do anything 248 times.
Image

operator[]
Posts: 156
Joined: Mon May 18, 2009 6:11 pm UTC
Location: Stockholm, Sweden

Re: Coding: Fleeting Thoughts

Postby operator[] » Sat Nov 16, 2013 5:37 pm UTC

Another approach to micro-optimization: precompute and bitpack the results of __builtin_parity:

Code: Select all

#include <bitset>
#include <vector>

uint32_t min_nonlinearity(const uint16_t* sbox) {
   const int LIM = 1 << 16;
   std::vector<std::bitset<LIM> > bits(LIM), bits2(LIM);
   for (int i = 1; i < LIM; i++) {
      for (int x = 0; x < LIM; x++) {
         bits[i][x] = __builtin_parity(x & i);
         bits2[i][x] = __builtin_parity(sbox[x] & i);
      }
   }

   int res = LIM;
   for (int i = 1; i < LIM; i++) {
      for (int j = 1; j < LIM; j++) {
         int nonlinearity = (bits[i] ^ bits2[j]).count();
         if (nonlinearity > (LIM / 2))
            nonlinearity = LIM - nonlinearity;
         res = std::min(res, nonlinearity);
      }
   }
   return res;
}

User avatar
Thesh
Made to Fuck Dinosaurs
Posts: 6568
Joined: Tue Jan 12, 2010 1:55 am UTC
Location: Colorado

Re: Coding: Fleeting Thoughts

Postby Thesh » Sat Nov 16, 2013 6:03 pm UTC

skeptical scientist wrote:I'm assuming __builtin_parity(n) is the number of 1s in the binary representation of n, mod 2?


Correct.

skeptical scientist wrote:I'm not sure if this is helpful, but one way to think of it is the following: for each fixed i, you have a collection of 16 vectors in F265536 (the nth vector has given by sbox[x]&(1 << (n-1)), and a target vector given by __builtin_parity(x&i). You want to find the nonzero linear combination of the first 16 vectors which best approximates the target vector. Right now you're doing this by brute force iterating over all 2^16-1 possible nonzero linear combinations (the for loop over j). There should be a better way. I don't remember off the top of my head, but I think algorithms for projecting onto subspaces work for vector spaces over finite fields? I think this will let you cut out one of your for loops, and may even speed up the overall algorithm by a few thousand times.


Yeah, I've been googling the problem, but I was hoping someone here had an idea to save me the time.

skeptical scientist wrote:1) How long does this take now?


Not sure, haven't had the patience to let it finish (over a couple hours at least).

skeptical scientist wrote:2) sbox appears to be a length 65536 array of uint16_t. Where does it come from? Can it be an arbitrary such array? Or do you know anything about it which you may be able to take advantage of in designing an algorithm?


It's randomly generated, contains every number from 0-65535 exactly once.

Xenomortis wrote:It takes a long time to do anything 248 times.

Pretty much, which is why I haven't been doing tiny optimizations; I need to reduce the number of loops if I want any hope of making this perform well.
Summum ius, summa iniuria.

Nyktos
Posts: 138
Joined: Mon Mar 02, 2009 4:02 pm UTC

Re: Coding: Fleeting Thoughts

Postby Nyktos » Sat Nov 16, 2013 9:39 pm UTC

ahammel wrote:I know I'm not the first one to complain about this, but I think Python's decision to move `reduce` to `functools` is really silly. If we want to make the point that comprehension expressions are to be preferred to the "higher order function and lambda" pattern, fine, but then why leave `map` and `filter`, which can be trivially replaced by a comprehension, and move `reduce`, which can't?
Map and filter were going to be removed, but people convinced Guido to keep them. Reduce was not so lucky because it leads to unreadable code in a way that the other HOFs don't, and because the new sum() builtin does what it was in practice most commonly used for.

If nothing else, forcing people to use an import for reduce might remind them that the operator module exists and typing "lambda x, y: x*y" is a terrible terrible thing.

User avatar
Xenomortis
Not actually a special flower.
Posts: 1446
Joined: Thu Oct 11, 2012 8:47 am UTC

Re: Coding: Fleeting Thoughts

Postby Xenomortis » Sat Nov 16, 2013 11:39 pm UTC

Just to get my head around the scale of it.
Thesh wrote:
skeptical scientist wrote:1) How long does this take now?


Not sure, haven't had the patience to let it finish (over a couple hours at least).

My computer is 7 hours into the following loop

Code: Select all

for (long i = 0 ; i < 65536L * 65536L * 65536L ; i++) ;

and is about 5% of the way there.
Image

User avatar
ahammel
My Little Cabbage
Posts: 2135
Joined: Mon Jan 30, 2012 12:46 am UTC
Location: Vancouver BC
Contact:

Re: Coding: Fleeting Thoughts

Postby ahammel » Sun Nov 17, 2013 1:42 am UTC

Nyktos wrote:
ahammel wrote:I know I'm not the first one to complain about this, but I think Python's decision to move `reduce` to `functools` is really silly. If we want to make the point that comprehension expressions are to be preferred to the "higher order function and lambda" pattern, fine, but then why leave `map` and `filter`, which can be trivially replaced by a comprehension, and move `reduce`, which can't?
Map and filter were going to be removed, but people convinced Guido to keep them.
So I've heard, but that's no explanation at all.

Reduce was not so lucky because it leads to unreadable code in a way that the other HOFs don't, and because the new sum() builtin does what it was in practice most commonly used for.
Well, yes, obviously discouraging stupid use cases of `reduce` is fine. What are the non-stupid use cases of `filter` and `map` that Guido wants to encourage by not banishing them to `functools`? As far as I can tell, they're entirely redundant with comprehension expressions.

Sidebar: I still think `reduce` is the best way to raise binary operations to n-ary functions. I hacked some ten lines out of a project the other day with this:

Code: Select all

def intersection(iterator_of_sets):
    return reduce(set.intersection, iterator_of_sets)
and I contend that this is more readable and less buggy than the alternatives that I could think up involving either a loop or the horrible:

Code: Select all

iterator_of_sets[0].intersection(iterator_of_sets[1:])


Speaking of the new sum builtin, it might be nice for the documentation to say what it actually does:
Python Docs wrote:Sums start and the items of an iterable from left to right and returns the total. start defaults to 0. The iterable‘s items are normally numbers, and the start value is not allowed to be a string.
So sum 'sums...the items of an iterable', eh? That's nice. What does it mean to 'sum' an iterable in this case? What if the iterable's items aren't numbers? Why isn't the start value allowed to be a string?

(I know the answers to these questions, by the way, but not from reading the docs.)

If nothing else, forcing people to use an import for reduce might remind them that the operator module exists and typing "lambda x, y: x*y" is a terrible terrible thing.
When I'm feeling less charitable, I wonder whether large parts of the Python functional programming community actually understand the benefits of functional programming, or whether typing `lambda` just makes them feel smart. I've read Python FP tutorials that didn't contain a word about why one would want to use FP idioms beyond "some people like it better".
He/Him/His/Alex
God damn these electric sex pants!

User avatar
You, sir, name?
Posts: 6983
Joined: Sun Apr 22, 2007 10:07 am UTC
Location: Chako Paul City
Contact:

Re: Coding: Fleeting Thoughts

Postby You, sir, name? » Sun Nov 17, 2013 11:21 am UTC

I'm increasingly frustrated with nulls in Java. Seems like 90%+ code-problems at work are null-related.

I've come to consider letting nulls enter or leave a function either as return value or object state* to be wrong. It's basically programming in C. Sure, you can do it--widely used and very stable projects/components are written in C/with null--but that is not reason to say passing/returning null is a good practice. You need so much extra process to prevent NPEs, because they can't be spotted by merely reading the code (they can often not even be detected with static code analysis), and having QA testing random things to see if they can trigger an NPE is hardly what I'd call a good way to get good code.

*if it isn't part of a builder pattern or something like that, and will eventually be populated.
I edit my posts a lot and sometimes the words wrong order words appear in sentences get messed up.

korona
Posts: 495
Joined: Sun Jul 04, 2010 8:40 pm UTC

Re: Coding: Fleeting Thoughts

Postby korona » Sun Nov 17, 2013 7:52 pm UTC

I have to defend Java in this respect. Java is not worse than C or C# here but certainly better than dynamically typed languages like Python. C++ is slightly better because it has immutable reference types that "should" never be null. However the language obviously doesn't guarantee that.

While it would be useful to have a modifier that declares a variable as non-null the null value solves more problems than it creates.
The policy "avoid null-valued parameters, return values and member state" is certainly bad. Often null is the best way to signal "this variable could a priori hold a value but doesn't because it makes no sense in the current state".

There are many cases where you want a function to take an optimal (object valued) parameter, or cases where you don't always have a well defined return value. Passing and returning a potential null reference is usually the best way to achieve that.

Consider the function java.util.Map<K, V>.get(K key). The function retrieves a value mapped to a given key or returns null if the key doesn't exist. You could ask why the function doesn't throw an exception when the key doesn't exist but the rationale behind this behavior is that it allows you to check the existence of items and retrieve them in a single call.

If you tend to run into many NPEs chances are that the pre- and post-conditions of your functions are not documented well enough. But a language like Java, C#, C++ or Python doesn't have the tools to properly enforce static pre- and post-conditions, so you have to ensure that as a programmer. Languages with richer type systems allow you to enforce more conditions statically, at the expense of more complicated code.

User avatar
Xanthir
My HERO!!!
Posts: 5400
Joined: Tue Feb 20, 2007 12:49 am UTC
Location: The Googleplex
Contact:

Re: Coding: Fleeting Thoughts

Postby Xanthir » Sun Nov 17, 2013 8:14 pm UTC

A nice solution that many languages (particular those with pattern matching) have taken is to have an Option or Maybe type instead of null. It's basically the same thing, but it's impossible to accidentally use a possibly-null value as plain value, as you have to explictly extract the plain value out of the Option.

With pattern matching this ends up being pretty elegant, as you just write both cases and produce a definitely value immediately. With functors and monads it's also elegant, as you can write code that just *assumes* it's a non-null value and then check the result at the end when it actually matters.
(defun fibs (n &optional (a 1) (b 1)) (take n (unfold '+ a b)))

User avatar
You, sir, name?
Posts: 6983
Joined: Sun Apr 22, 2007 10:07 am UTC
Location: Chako Paul City
Contact:

Re: Coding: Fleeting Thoughts

Postby You, sir, name? » Sun Nov 17, 2013 8:20 pm UTC

I'm not saying you should never use null, I'm saying you should use them as sparingly as possible, and when you do, encapsulate them so that they don't leak into your business logic (by using for example factories or builders to contain an unfinished). I'm fully aware that it's not really feasible to do away completely with nulls as that would make a lot of things completely impossible (such as circular references).

Maps are an excellent example of a "but it works in C"-type argument. Maps are looking the way they do is because they are a very old part of the java standard library. They could just as easily return something like

Code: Select all

class MapEntry<T> {
  private final T value;
  public MapEntry(T value) {
    this.value = value;
  }
  public MapEntry() {
    this.value = null;
  }
  public boolean exists() {
     return value != null;
  }
  public T get() {
     if (!exists()) throw new InvalidEntryException();
     return value;
  }
}

... which has the advantage of failing immediately when you try to mess around with it the wrong way, rather than eventually like nulls do. Furthermore, you can tell from the function prototype "MapEntry<T> get(K key)" this function does not return an object of type T, but something that may be an object of the type T. Returning this type of construct instead of a raw reference has virtually zero impact on performance, and preserves map usage patterns, yet renders the Map object completely incapable of introducing a NPE into your code.

Functions that accept null as arguments should be two functions, one that takes an argument and one that does not.

The problems with returning null is that I can't tell if you are returning null (or passing null) by looking at the code. I have to examine the code I'm calling, and quite possibly the code it is calling, so on and so forth.
I edit my posts a lot and sometimes the words wrong order words appear in sentences get messed up.

korona
Posts: 495
Joined: Sun Jul 04, 2010 8:40 pm UTC

Re: Coding: Fleeting Thoughts

Postby korona » Sun Nov 17, 2013 8:41 pm UTC

Returning a MapEntry<T> doesn't make sense because it involves one memory allocation and object creation per call.
Surrounding calls with if(result.exists()) is MUCH uglier than simply doing if(result == null). The only thing it does is making it explicit if a function can return a null value. But it adds a meaningless class definition, boilerplate code and runtime overhead. Not to mention that it complicated syntax with basically no benefit. map.get(key).get() just looks incredibly convoluted. I consider such a construct very bad style and would certainly reject such a commit to the codebases I work with.

Having two functions that do the same thing and differ only in a possibly-null argument is also very alarming because of code duplication. Sometimes it may be possible to introduce two functions but there are many cases where it makes absolutely no sense.

The "I have to look at the function I'm calling to determine if it can return null" is an incredibly stupid argument. You have to look at the function that you call anyway because well, you want to know what it does. Also the specification whether a function can return null should be part of the function's interface documentation. You don't have to look at the function's code to determine that. In case of Java you have the excellent tool JavaDoc that exists for exactly that purpose.

User avatar
You, sir, name?
Posts: 6983
Joined: Sun Apr 22, 2007 10:07 am UTC
Location: Chako Paul City
Contact:

Re: Coding: Fleeting Thoughts

Postby You, sir, name? » Sun Nov 17, 2013 9:04 pm UTC

korona wrote:Returning a MapEntry<T> doesn't make sense because it involves one memory allocation and object creation per call.


Map lookups are inherently expensive, adding a short lived memory allocation to that is pissing in the ocean, especially an object like MapEntry which is highly JIT-friendly (provided it doesn't leave the function that does the getting).

korona wrote:Surrounding calls with if(result.exists()) is MUCH uglier than simply doing if(result == null). The only thing it does is making it explicit if a function can return a null value. But it adds a meaningless class definition, boilerplate code and runtime overhead. Not to mention that it complicated syntax with basically no benefit.


The class definition is internal to "Map", and you need to write approximately the same amount of code regardless of which error handling strategy you use. The benefit is that the code is infinitely more robust.

korona wrote:map.get(key).get() just looks incredibly convoluted.


What the hell? It's also very dangerous and should never ever be done. That's like map.get(key).doStuff(). You need to existence-check when you get from a map, or you're going to have a bad time. You usage pattern with the MapEntry object would look something along the lines of:

Code: Select all

String findAddressOfPerson(String person) {
  MapEntry<String> addressEntry = addressBook.get(person);
  if (addressEntry.exists()) {
    return addressEntry.get();
  } else {
    return "";
  }
}


korona wrote:Having two functions that do the same thing and differ only in a possibly-null argument is also very alarming because of code duplication. Sometimes it may be possible to introduce two functions but there are many cases where it makes absolutely no sense.


A function should have a single responsibility. If its behavior changes based on whether its input is null, it has too many responsibilities and should be two functions. Code duplication will only be a problem if the function is already unreasonably long, in which case, maybe the optional parameter may be gotten rid of by breaking the function down into sequential rather than parallel steps.

korona wrote:The "I have to look at the function I'm calling to determine if it can return null" is an incredibly stupid argument. You have to look at the function that you call anyway because well, you want to know what it does.


In object oriented code, you generally speaking can't guarantee that the function you're looking at is the function you will call. Polymorphism is a bitch that way.

Anyway, the problem with NPEs is that there is a disjoint in cause and effect -- a field may be set to null by a function, and then 20 minutes later, someone tries to use that field and kablam! So you're stuck with the detective game, trying to piece together possible ways why this field managed to get set to zero, sifting through references and call hierarchies. All this is often time consuming because of the aforementioned difficulty in determining if a code calls code that returns null by reading it. If on the other hand something like MapEntry had encapsulated the null, the code would have blown up on the very line someone attempted direct access to a null variable, and debugging would take seconds rather than hours.
I edit my posts a lot and sometimes the words wrong order words appear in sentences get messed up.

korona
Posts: 495
Joined: Sun Jul 04, 2010 8:40 pm UTC

Re: Coding: Fleeting Thoughts

Postby korona » Sun Nov 17, 2013 9:48 pm UTC

You, sir, name? wrote:
korona wrote:Returning a MapEntry<T> doesn't make sense because it involves one memory allocation and object creation per call.


Map lookups are inherently expensive, adding a short lived memory allocation to that is pissing in the ocean, especially an object like MapEntry which is highly JIT-friendly (provided it doesn't leave the function that does the getting).

Hash lookups are incredibly fast. At least much faster than object creation + related gc cost. A fast-path hash lookup takes about 2 cpu instructions. The fastest implementation of object creation + initialization takes at least a dozen instructions even for copying GCs, possibly memory barriers and inter-cpu synchronization. And that doesn't even include garbage collection cost.

Returning objects is never JIT friendly. The JIT either has to do a polymorphic type inference pass + inline pass or a return-object-on-stack optimization to get rid of the object allocation and both are neither trivial nor inexpensive. If the code is not a hot spot the object WILL be created. It may be a small overhead but it still is a unnecessary one.

korona wrote:Surrounding calls with if(result.exists()) is MUCH uglier than simply doing if(result == null). The only thing it does is making it explicit if a function can return a null value. But it adds a meaningless class definition, boilerplate code and runtime overhead. Not to mention that it complicated syntax with basically no benefit.


The class definition is internal to "Map", and you need to write approximately the same amount of code regardless of which error handling strategy you use. The benefit is that the code is infinitely more robust.

More robust against programmers that don't read the interface documentation. "the function returns x if y and null otherwise" is the most basic post-condition that a function can have. You cannot invent a new class for every possible condition on return values. What do you do if a function can only return positive numbers or take them as arguments? Invent a CheckedPositiveInteger class? What if it can only take primes? Invent a CheckedPrime class with a O(n) integer retrieval function that performs a runtime sanity check?

Java and similar languages (e.g. C, C++, Python) lack the tools to properly express conditions on variables. It is your job to express them. The best way to do that is not adding meaningless runtime checks but writing proper documentation. Languages like Clojure and Python have support for runtime checked pre- and post-conditions. Haskell and ML variants have type systems that allow you to express some static conditions. Mathematical languages like Coq allow you to express arbitrary statically checked conditions at the expense of a much more complicated language.

korona wrote:map.get(key).get() just looks incredibly convoluted.


What the hell? It's also very dangerous and should never ever be done. That's like map.get(key).doStuff(). You need to existence-check when you get from a map, or you're going to have a bad time.
korona wrote:Having two functions that do the same thing and differ only in a possibly-null argument is also very alarming because of code duplication. Sometimes it may be possible to introduce two functions but there are many cases where it makes absolutely no sense.


A function should have a single responsibility. If its behavior changes based on whether its input is null, it has too many responsibilities and should be two functions. Code duplication will only be a problem if the function is already unreasonably long, in which case, maybe the optional parameter may be gotten rid of by breaking the function down into sequential rather than parallel steps.

There are many cases where you know that the map contains the key you are looking for.
Breaking all functions into basic ones is not realistic. Real life code cannot always be broken into small simple chunks.
Both arguments are a cheap excuse for not reading the interface documentation.

korona wrote:The "I have to look at the function I'm calling to determine if it can return null" is an incredibly stupid argument. You have to look at the function that you call anyway because well, you want to know what it does.


In object oriented code, you generally speaking can't guarantee that the function you're looking at is the function you will call. Polymorphism is a bitch that way.

Polymorphic overrides still have to fulfill the contract specified by the base class. That is not a problem.

Anyway, the problem with NPEs is that there is a disjoint in cause and effect -- a field may be set to null by a function, and then 20 minutes later, someone tries to use that field and kablam... so you're stuck with the detective game, trying to piece together possible ways why this field managed to get set to zero, sifting through references and call hierarchies. Which is often time consuming because of the aforementioned difficulty in determining if a code calls code that returns null by reading it. If on the other hand something like MapEntry had encapsulated the null, the code would have blown up on the very line someone attempted direct access to a null variable, and debugging would take seconds rather than hours.

You are just moving the check to the MapEntry class. You'll get the same problems when code starts to pass around MapEntry values. The right way is to look at the documentation of the function you're calling and observe "hey, this function can return null, but that means something in my code messed up so I throw an exception".

EDIT:
Your example code:

Code: Select all

String findAddressOfPerson(String person) {
  MapEntry<String> addressEntry = addressBook.get(person);
  if (addressEntry.exists()) {
    return addressEntry.get();
  } else {
    return "";
  }
}

is an example for very bad style. The entire function can be replaced by addressBook.get(person). That is not only much less verbose but also more correct: Instead of returning null or throwing an exception your code returns an illegal, arbitrary and ambiguous return value namely the empty string!

Nyktos
Posts: 138
Joined: Mon Mar 02, 2009 4:02 pm UTC

Re: Coding: Fleeting Thoughts

Postby Nyktos » Mon Nov 18, 2013 12:27 am UTC

ahammel wrote:
Reduce was not so lucky because it leads to unreadable code in a way that the other HOFs don't, and because the new sum() builtin does what it was in practice most commonly used for.
Well, yes, obviously discouraging stupid use cases of `reduce` is fine. What are the non-stupid use cases of `filter` and `map` that Guido wants to encourage by not banishing them to `functools`? As far as I can tell, they're entirely redundant with comprehension expressions.
Generators have a lot of overhead; map and filter don't, and are quite fast by comparison. So that makes them sort of not-redundant, though of course the entire itertools module could be moved to builtins with that logic. I can't seem to find the exact reasoning for keeping them built-in, but as I understand it it's basically that people found them to be more readable than generator expressions when just applying an already-defined named function.

I use them sometimes myself when I feel they make the code cleaner than a genexpr would (certainly never with a lambda), but not so often that I'd really complain about having to import them from itertools.

User avatar
Yakk
Poster with most posts but no title.
Posts: 11115
Joined: Sat Jan 27, 2007 7:27 pm UTC
Location: E pur si muove

Re: Coding: Fleeting Thoughts

Postby Yakk » Mon Nov 18, 2013 4:04 am UTC

The "type that can be there" (optional<T> or maybe<T>), and the "type that is there", are useful types. Java lacks both, because everything can be *not there* (everything is a pointer), and because everything can be not there, optional<T> looks redundant.

Modern C++ encourages value semantics when possible, type erasure and pImpl when not, and pointers as a last resort. optional<T> works beautifully, as does the newly proposed "expected" optional-or-excuse (expected<int, error_code>). Explicit exceptions (or error codes) you can handle explicitly or implicitly as you choose!
One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision - BR

Last edited by JHVH on Fri Oct 23, 4004 BCE 6:17 pm, edited 6 times in total.

User avatar
ahammel
My Little Cabbage
Posts: 2135
Joined: Mon Jan 30, 2012 12:46 am UTC
Location: Vancouver BC
Contact:

Re: Coding: Fleeting Thoughts

Postby ahammel » Mon Nov 18, 2013 4:38 am UTC

Nyktos wrote:Generators have a lot of overhead; map and filter don't, and are quite fast by comparison. So that makes them sort of not-redundant, though of course the entire itertools module could be moved to builtins with that logic. I can't seem to find the exact reasoning for keeping them built-in, but as I understand it it's basically that people found them to be more readable than generator expressions when just applying an already-defined named function.
Huh, yeah, it is faster. It's slower if you need the intermediate list, though.

Code: Select all

┌─────[alex@asimov] (~)
└─╼ python --version
Python 3.3.2
┌─────[alex@asimov] (~)
└─╼ python -m timeit "less_than_3 = lambda x: x < 3" "ls = [1, 2, 3, 4, 5]" "filter(less_than_3, ls)"                                                                   
1000000 loops, best of 3: 1.65 usec per loop
┌─────[alex@asimov] (~)
└─╼ python -m timeit "less_than_3 = lambda x: x < 3" "ls = [1, 2, 3, 4, 5]" "(x for x in ls if less_than_3(x))"                                                         
100000 loops, best of 3: 4.18 usec per loop
┌─────[alex@asimov] (~)
└─╼ python -m timeit "less_than_3 = lambda x: x < 3" "ls = [1, 2, 3, 4, 5]" "f = filter(less_than_3, ls)" "for n in f: pass"
100000 loops, best of 3: 6.4 usec per loop
┌─────[alex@asimov] (~)
└─╼ python -m timeit "less_than_3 = lambda x: x < 3" "ls = [1, 2, 3, 4, 5]" "f = (x for x in ls if less_than_3(x))" "for n in f: pass"
100000 loops, best of 3: 8.67 usec per loop
┌─────[alex@asimov] (~)
└─╼ python -m timeit "less_than_3 = lambda x: x < 3" "ls = [1, 2, 3, 4, 5]" "list(filter(less_than_3, ls))"                                                             
100000 loops, best of 3: 11.9 usec per loop
┌─────[alex@asimov] (~)
└─╼ python -m timeit "less_than_3 = lambda x: x < 3" "ls = [1, 2, 3, 4, 5]" "[x for x in ls if less_than_3(x)]"     
100000 loops, best of 3: 6.23 usec per loop


I don't buy that `reduce` is any less readable than `map` and `filter` for the single named function case, to tell you the truth.
He/Him/His/Alex
God damn these electric sex pants!

User avatar
TheChewanater
Posts: 1279
Joined: Sat Aug 08, 2009 5:24 am UTC
Location: lol why am I still wearing a Santa suit?

Re: Coding: Fleeting Thoughts

Postby TheChewanater » Mon Nov 18, 2013 5:09 am UTC

I recently realized that my current project would be incredibly simplified if I had an open source units and measures library (like JScience or Boost.Units) for both C++ and Java with the same API. Kind of a shot in the dark, but does anyone happen to know of a solution? I might just make my own and base it on the JScience documentation.
ImageImage
http://internetometer.com/give/4279
No one can agree how to count how many types of people there are. You could ask two people and get 10 different answers.

Nyktos
Posts: 138
Joined: Mon Mar 02, 2009 4:02 pm UTC

Re: Coding: Fleeting Thoughts

Postby Nyktos » Mon Nov 18, 2013 11:11 am UTC

ahammel wrote:I don't buy that `reduce` is any less readable than `map` and `filter` for the single named function case, to tell you the truth.
Readable maybe, but if the function isn't associative, it can still require some mental gymnastics to understand. (And in the case that it is, it's probably coming from operator, so need an import either way.)

User avatar
Xenomortis
Not actually a special flower.
Posts: 1446
Joined: Thu Oct 11, 2012 8:47 am UTC

Re: Coding: Fleeting Thoughts

Postby Xenomortis » Mon Nov 18, 2013 3:30 pm UTC

I just spent half an hour trying to figure out why LINQ wasn't working.
Then I remembered the site I'm working on uses .NET 2.0 :(
Image

User avatar
You, sir, name?
Posts: 6983
Joined: Sun Apr 22, 2007 10:07 am UTC
Location: Chako Paul City
Contact:

Re: Coding: Fleeting Thoughts

Postby You, sir, name? » Mon Nov 18, 2013 5:32 pm UTC

korona wrote:
korona wrote:Surrounding calls with if(result.exists()) is MUCH uglier than simply doing if(result == null). The only thing it does is making it explicit if a function can return a null value. But it adds a meaningless class definition, boilerplate code and runtime overhead. Not to mention that it complicated syntax with basically no benefit.


The class definition is internal to "Map", and you need to write approximately the same amount of code regardless of which error handling strategy you use. The benefit is that the code is infinitely more robust.


More robust against programmers that don't read the interface documentation. "the function returns x if y and null otherwise" is the most basic post-condition that a function can have. You cannot invent a new class for every possible condition on return values. What do you do if a function can only return positive numbers or take them as arguments? Invent a CheckedPositiveInteger class? What if it can only take primes? Invent a CheckedPrime class with a O(n) integer retrieval function that performs a runtime sanity check?


The scenario of returning null should be pretty rare. If your code bases passes null regularly, I'd hate to be the guy in charge of maintaining it. Nothing is harder to read than a function where every other line is a null-check.

korona wrote:Java and similar languages (e.g. C, C++, Python) lack the tools to properly express conditions on variables. It is your job to express them. The best way to do that is not adding meaningless runtime checks but writing proper documentation. Languages like Clojure and Python have support for runtime checked pre- and post-conditions. Haskell and ML variants have type systems that allow you to express some static conditions. Mathematical languages like Coq allow you to express arbitrary statically checked conditions at the expense of a much more complicated language.


Documentation is not validated by the compiler, or the JVM. It's just a bunch of words that may or may not accurately describe the code.

korona wrote:Breaking all functions into basic ones is not realistic. Real life code cannot always be broken into small simple chunks.

Now who is spouting cheap excuses?! Virtually any code can be written in a terse and clean fashion. Usually doesn't take more than a couple of hours to convert a horrible 3000+ line class full of enterprise spaghetti code to something readable and nice.

korona wrote:Both arguments are a cheap excuse for not reading the interface documentation.

It's a damage reduction strategy. if you don't let nulls contaminate your state, you don't get null pointer exceptions. My argument is basically the same as having arrays automatically bounds check access. That's just as much a cheap excuse for not manually bounds checking every array index.

korona wrote:
korona wrote:The "I have to look at the function I'm calling to determine if it can return null" is an incredibly stupid argument. You have to look at the function that you call anyway because well, you want to know what it does.


In object oriented code, you generally speaking can't guarantee that the function you're looking at is the function you will call. Polymorphism is a bitch that way.

Polymorphic overrides still have to fulfill the contract specified by the base class. That is not a problem.


... until a function is buggy, and returns a null.

korona wrote:You are just moving the check to the MapEntry class. You'll get the same problems when code starts to pass around MapEntry values. The right way is to look at the documentation of the function you're calling and observe "hey, this function can return null, but that means something in my code messed up so I throw an exception".


This is fine. The MapEntry class means "this may have a value, you should always check", versus the raw type which should mean "this has a value". Even though it's sort of questionable practice ot pass on MapEntry, they retain their semantics.

korona wrote:is an example for very bad style. The entire function can be replaced by addressBook.get(person). That is not only much less verbose but also more correct: Instead of returning null or throwing an exception your code returns an illegal, arbitrary and ambiguous return value namely the empty string!


The empty string is not an arbitrary value. Returning values like empty lists and empty strings instead of null is a fairly common practice. Clean Code recommends the practice of returning empty objects over null, along with avoiding passing and returning nulls altogether.

The reason java.util.Map doesn't return "" is because java generics are pitiful and weak, and can't do that (c.f. std::map::operator[], which *will*).
I edit my posts a lot and sometimes the words wrong order words appear in sentences get messed up.

User avatar
Yakk
Poster with most posts but no title.
Posts: 11115
Joined: Sat Jan 27, 2007 7:27 pm UTC
Location: E pur si muove

Re: Coding: Fleeting Thoughts

Postby Yakk » Mon Nov 18, 2013 6:44 pm UTC

The problem in Java isn't the lack of ability to return "this could be a T or maybe not" but rather the lack of a "this is a T". This leads everyone to use "this could be a T, or maybe not" as the type they use when they mean "this is a T".

And as many many more functions return "this is a T" than "this could be a T, or maybe not", and because it takes a bunch of code to safely check "is this a T or not", and treating "this could be T, or maybe not" as "this is a T" works really well if it is rarely not-a-T, you get entire code bases that treat functions that return "this could be a T, or maybe not" like "this is a T" and only fail on corner cases. Which is a code maintenance nightmare.

The "solution" of check the documentation of each function to determine if the "this could be a T, or maybe not" on each and every function you call to determine if it is "this is a T" or if it really is "this could be a T, or maybe not" is just another code maintenance nightmare, especially when the language lacks any support for enforcing such.

Code that works 99.9% of the time and fails horribly 0.1%, and distinguishing it from code that works 100% of the time requires reading the source and documentation of every line in the code, is a code maintenance nightmare. You need a brand new code base with rigorous code quality standards and little deadline pressure, a huge huge investment in testing to catch every such case, or entire teams full of perfect coders.

Better than nothing would be such code should be obviously different at the point of use, so you can look at the code and say "oh, they missed the failure branch". Better than that would be the compiler/code lint tool spotting such cases for you.
One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision - BR

Last edited by JHVH on Fri Oct 23, 4004 BCE 6:17 pm, edited 6 times in total.

troyp
Posts: 557
Joined: Thu May 22, 2008 9:20 pm UTC
Location: Lismore, NSW

Re: Coding: Fleeting Thoughts

Postby troyp » Mon Nov 18, 2013 7:13 pm UTC

ahammel wrote:I know I'm not the first one to complain about this, but I think Python's decision to move `reduce` to `functools` is really silly. If we want to make the point that comprehension expressions are to be preferred to the "higher order function and lambda" pattern, fine, but then why leave `map` and `filter`, which can be trivially replaced by a comprehension, and move `reduce`, which can't?

Guido apparently can't understand nontrivial usage of reduce, and rather than learn, he's decared the function to be unreadable.

Well, yes, obviously discouraging stupid use cases of `reduce` is fine. What are the non-stupid use cases of `filter` and `map` that Guido wants to encourage by not banishing them to `functools`? As far as I can tell, they're entirely redundant with comprehension expressions.

Except that comprehensions aren't first-class. It's not all that rare for me to pass map or filter as an argument.

Sidebar: I still think `reduce` is the best way to raise binary operations to n-ary functions. I hacked some ten lines out of a project the other day with this:

Code: Select all

def intersection(iterator_of_sets):
    return reduce(set.intersection, iterator_of_sets)
and I contend that this is more readable and less buggy than the alternatives that I could think up involving either a loop or the horrible:

Code: Select all

iterator_of_sets[0].intersection(iterator_of_sets[1:])

I completely agree that reduce can be much more readable than general iteration or recursion. A fold is one of the basic HOFs, and as such, when you see one, you only have to "fill in the blanks". With with a loop you have to read and understand the entire thing*. When you can use a construct which more specific (while still being familiar), you should, in general, do so**.

* I often suspect that people assess readability "by the line" rather than "by the logical unit". Ie. they judge a concise construct unfavourably if it takes them 3 times as long to read as an average line, even if it actually replaces 10 of those average lines.
** I don't actually follow this religiously, but I agree with it as a rule of thumb.


Yakk wrote:Modern C++ encourages value semantics when possible, type erasure and pImpl when not, and pointers as a last resort. optional<T> works beautifully, as does the newly proposed "expected" optional-or-excuse (expected<int, error_code>).

I was only just wishing for a C++ Option type! That's awesome. But cppreference.com says it's voted out of C++14. Do you know why?

korona
Posts: 495
Joined: Sun Jul 04, 2010 8:40 pm UTC

Re: Coding: Fleeting Thoughts

Postby korona » Mon Nov 18, 2013 7:27 pm UTC

The scenario of returning null should be pretty rare. If your code bases passes null regularly, I'd hate to be the guy in charge of maintaining it. Nothing is harder to read than a function where every other line is a null-check.

I never had such a problem. In our code base we always have documentation like "this functions returns null iff x". Because the documentation is part of the source it is easy to keep it up to date.
If functions cannot return null you don't need a null check. If they can return null returning a boxed value doesn't save you any code.

Documentation is not validated by the compiler, or the JVM. It's just a bunch of words that may or may not accurately describe the code.

Sure. I agree that it would be very nice to have a syntax that is automatically verified by the compiler.
But I don't think boxing values is the solution in languages that don't have such syntax. I think null checking values that can be null is a better solution. As I said, being not-null is the most basic condition that you can have. You cannot handle every condition by introducing a boxing class.

Virtually any code can be written in a terse and clean fashion. Usually doesn't take more than a couple of hours to convert a horrible 3000+ line class full of enterprise spaghetti code to something readable and nice.

That might be true sometimes but sometimes there are algorithms that cannot be implemented in less than 300 lines and that take optional arguments.

It's a damage reduction strategy. if you don't let nulls contaminate your state, you don't get null pointer exceptions. My argument is basically the same as having arrays automatically bounds check access. That's just as much a cheap excuse for not manually bounds checking every array index.

The primary reason why Java has automatic array bounds checking is not because it is convenient for the programmer. The reason is that it is a necessary evil which we have to accept in order to guarantee memory safety. Most of the time I do not want automatic bounds checking. But it is required by the JVM and that makes sense from the JVMs point of view.

Maybe the fundamental difference between your coding standard and mine is that I don't think of Java references to a type T as "this is a value of type T" but rather "this might be a value of type T" i.e. a safe C-style pointer. I think of Java as a language to use instead of C when I want the ability to run untrusted code or platform independence. I don't think of Java as a language on such a high level as Python, JavaScript, C++ (to some extend) etc.

... until a function is buggy, and returns a null.

Functions don't magically return null. The new operator always returns a valid reference. I know that there are many bugs that even survive elaborate testing and reviews. But null returns usually don't. Just think of Java references the same way you think about C pointers and you'll be fine.

This is fine. The MapEntry class means "this may have a value, you should always check", versus the raw type which should mean "this has a value". Even though it's sort of questionable practice ot pass on MapEntry, they retain their semantics.

The problem I was referring to (and the one you mentioned in your post) is you get deferred exceptions.

The empty string is not an arbitrary value. Returning values like empty lists and empty strings instead of null is a fairly common practice. Clean Code recommends the practice of returning empty objects over null, along with avoiding passing and returning nulls altogether.

I agree with returning empty lists, sets and maps because they make more sense semantically. But I'd be careful when using the empty string. Depending on the configuration the empty string could also mean that the user didn't enter an address.

The reason java.util.Map doesn't return "" is because java generics are pitiful and weak, and can't do that (c.f. std::map::operator[], which *will*).

std::map::operator[] has insert-if-not-exists semantics. The equivalent to Java's .get() is std::map::find() which returns an past-end iterator when the key is not in the map. C++ iterators are essentially generalized C pointers and the past-end iterator corresponds to Java's null in this context. And returning boxed objects (e.g. iterators) in C++ is fine because C++ has much better tools for object construction and destruction, type casting and operator overloading.

User avatar
Xanthir
My HERO!!!
Posts: 5400
Joined: Tue Feb 20, 2007 12:49 am UTC
Location: The Googleplex
Contact:

Re: Coding: Fleeting Thoughts

Postby Xanthir » Mon Nov 18, 2013 8:23 pm UTC

korona wrote:I know that there are many bugs that even survive elaborate testing and reviews. But null returns usually don't. Just think of Java references the same way you think about C pointers and you'll be fine.

I'm not sure how you managed to work only on magical codebases that don't have NPEs, but I assure that in most of the world, NPEs (in Java) and use-after-frees (in C) are very common.
(defun fibs (n &optional (a 1) (b 1)) (take n (unfold '+ a b)))

korona
Posts: 495
Joined: Sun Jul 04, 2010 8:40 pm UTC

Re: Coding: Fleeting Thoughts

Postby korona » Mon Nov 18, 2013 8:39 pm UTC

I'm not saying NPEs don't happen. That is why I wrote the first sentence you quoted.
But buggy functions randomly returning null is not that common. And arguably not as bad as a buggy function returning another random value.

I also agree that deferred NPEs are a huge problem for debugging and monitoring running applications. I just think boxing values is not the right approach to this problem.

-------

Anyways here is something else I thought I'd post here. I've spend the last few days to work out a typed lambda calculus that I'll use as the backend for a programming language.

It is strongly normalizing (i.e. not Turing complete but all computations terminate) and has a rich type system. From a programming language point of view it is pure (i.e. no expressions have side effects or mutable state) and safe (guaranteed not to access undefined memory locations). It includes first class types (i.e. you can write a function that returns a type and instantiate a variable with that type) and dependent types (i.e. you can have a function where the result type depends on the argument. Consider the function that maps an integer n to the zero vector in R^n). It is similar to Coq in this respect but there are conceptual differences. Type checking can be done in linear time.

Here is the first problem from Project Euler in this calculus:

Code: Select all

listFold[listFilter[range[1000] ?x:Int logicOr[intEq[intMod[$x:Int 3] 0] intEq[intMod[$x:Int 5] 0]]] ?x:Int ?r:Int intAdd[$x:Int $r:Int] 0]

Variables are denoted by $name:Type. Builtin functions are denoted by function[arg1 arg2 ... argN]. Lambda abstraction has the syntax ?variable:Type expr.

Here is a more interesting example: The proof that x - x = 0 for integers x, from the axiom x + (0 - x) = 0 and the definition of subtraction x - y = x + (0 - y).

Code: Select all

&intEq[?y:Int intEq[intSub[$x:Int $x:Int] $y:Int] intAdd[$x:Int intSub[0 $x:Int]] 0 &univInstance[&univInstance[#intSubDefn $x:Int] $x:Int] &univInstance[#intAddInverse $x:Int]]

Axioms are denoted by #axiom. The axioms used in this proof have the following type:
#intAddInverse: Proof[logicForall[?x:Int intEq[intAdd[$x:Int intSub[0 $x:Int]] 0]]]
#intSubDefn: Proof[logicForall[?x:Int logicForall[?y:Int intEq[intSub[$x:Int $y:Int] intAdd[$x:Int intSub[0 $y:Int]]]]]]

Inference rules are denoted by &rule.
&univInstance is univeral instantiation and has type (I'm using currying notation here): Proof[logicForall[?x:T p(x)]] -> c:T -> Proof[p(c)].
&intEq is Leibniz's law for integer equality (i.e. the fact that if I have a proposition p(x) and x = y then p(y) holds). It has type: p:(T -> Bool) -> u:T -> v:T -> Proof[p(u)] -> Proof[intEq[u, v]] -> Proof[p(v)].

Taken together that means that we can infer the following types:
&univInstance[#intAddInverse $x:Int]] is of type: Proof[intEq[intAdd[$x:Int intSub[0 $x:Int]] 0]]
&univInstance[&univInstance[#intSubDefn $x:Int] $x:Int] is of type: Proof[intEq[intSub[$x:Int $x:Int] intAdd[$x:Int intSub[0 $x:Int]]]]
so the whole expression is of type: Proof[intEq[intSub[$x:Int $x:Int] 0]]

The fact that proofs naturally integrate with code means that I can statically verify arbitary conditions on values. For example the division builtin takes a proof that the divisor is non-zero as argument. List access builtins take proofs that the index is in range. Depending on the use case one can supply such proofs or do the necessary checks at runtime (which is easier ofc but incurs a runtime cost). Integration with automated theorem provers could allow you to perform the proofs mechanically.

Now that this calculus is working (i.e. I have a working parser and evaluator) I'm planning to:
1) Add a generalization of Haskell's IO type that allows arbitrary computations in this language
2) Develop a programming language that compiles to this calculus. While the calculus is conceptually clean it is hard to work with as a human. Ideally the language should look like languages programmers are used to but allow all manipulations the calculus itself allows.
3) Write a self-certifying compiler in that language to compile the calculus to assembly.
4) ???
5) Profit!

lgw
Posts: 437
Joined: Mon Apr 12, 2010 10:52 pm UTC

Re: Coding: Fleeting Thoughts

Postby lgw » Tue Nov 19, 2013 12:53 am UTC

Xanthir wrote:
korona wrote:I know that there are many bugs that even survive elaborate testing and reviews. But null returns usually don't. Just think of Java references the same way you think about C pointers and you'll be fine.

I'm not sure how you managed to work only on magical codebases that don't have NPEs, but I assure that in most of the world, NPEs (in Java) and use-after-frees (in C) are very common.


I've always liked the idea of complier-enforced contracts in a language. Such that you can declare your function to never return null, and the compiler will error if (by static analysis) it's possible to return null along any branch. Not perfect, since static analysis can only go so far, but really quite helpful. The idea never really caught on in mainstream programming languages, however.
"In no set of physics laws do you get two cats." - doogly

User avatar
Aaeriele
Posts: 2127
Joined: Tue Feb 23, 2010 3:30 am UTC
Location: San Francisco, CA

Re: Coding: Fleeting Thoughts

Postby Aaeriele » Tue Nov 19, 2013 8:06 am UTC

ahammel wrote:Sidebar: I still think `reduce` is the best way to raise binary operations to n-ary functions. I hacked some ten lines out of a project the other day with this:

Code: Select all

def intersection(iterator_of_sets):
    return reduce(set.intersection, iterator_of_sets)
and I contend that this is more readable and less buggy than the alternatives that I could think up involving either a loop or the horrible:

Code: Select all

iterator_of_sets[0].intersection(iterator_of_sets[1:])


Code: Select all

set.intersection(*iterator_of_sets)

is both shorter and more concise.

troyp wrote:Except that comprehensions aren't first-class. It's not all that rare for me to pass map or filter as an argument.

Argument to what?
Vaniver wrote:Harvard is a hedge fund that runs the most prestigious dating agency in the world, and incidentally employs famous scientists to do research.

afuzzyduck wrote:ITS MEANT TO BE FLUTTERSHY BUT I JUST SEE AAERIELE! CURSE YOU FORA!

User avatar
Xanthir
My HERO!!!
Posts: 5400
Joined: Tue Feb 20, 2007 12:49 am UTC
Location: The Googleplex
Contact:

Re: Coding: Fleeting Thoughts

Postby Xanthir » Tue Nov 19, 2013 4:13 pm UTC

Aaeriele wrote:
troyp wrote:Except that comprehensions aren't first-class. It's not all that rare for me to pass map or filter as an argument.

Argument to what?

To other functions which take a callback argument, presumably.
(defun fibs (n &optional (a 1) (b 1)) (take n (unfold '+ a b)))

User avatar
Aaeriele
Posts: 2127
Joined: Tue Feb 23, 2010 3:30 am UTC
Location: San Francisco, CA

Re: Coding: Fleeting Thoughts

Postby Aaeriele » Tue Nov 19, 2013 7:41 pm UTC

Xanthir wrote:
Aaeriele wrote:
troyp wrote:Except that comprehensions aren't first-class. It's not all that rare for me to pass map or filter as an argument.

Argument to what?

To other functions which take a callback argument, presumably.


Right, I'm just trying to envision a common scenario in which you'd want to do that where there isn't a clearer way.
Vaniver wrote:Harvard is a hedge fund that runs the most prestigious dating agency in the world, and incidentally employs famous scientists to do research.

afuzzyduck wrote:ITS MEANT TO BE FLUTTERSHY BUT I JUST SEE AAERIELE! CURSE YOU FORA!

troyp
Posts: 557
Joined: Thu May 22, 2008 9:20 pm UTC
Location: Lismore, NSW

Re: Coding: Fleeting Thoughts

Postby troyp » Wed Nov 20, 2013 3:31 pm UTC

Aaeriele wrote:
ahammel wrote:Sidebar: I still think `reduce` is the best way to raise binary operations to n-ary functions. I hacked some ten lines out of a project the other day with this:

Code: Select all

def intersection(iterator_of_sets):
    return reduce(set.intersection, iterator_of_sets)
and I contend that this is more readable and less buggy than the alternatives that I could think up involving either a loop or the horrible:

Code: Select all

iterator_of_sets[0].intersection(iterator_of_sets[1:])


Code: Select all

set.intersection(*iterator_of_sets)

is both shorter and more concise.

That's only a criticism of his example, though. You're pointing out that intersection is already an n-ary function. If it actually was a binary operator, reduce would probably be your best bet to convert it. (Ideally, it'd be nice to have something more specific, like having + and * overloaded on sets and then a product function you could use on anything supporting multiplication.)

troyp wrote:Except that comprehensions aren't first-class. It's not all that rare for me to pass map or filter as an argument.

Argument to what?

Actually...come to think of it, I probably don't use them in a way I couldn't use comprehensions - not in Python. I think usually when I pass map and filter around, they're partially applied. Having them as functions makes that very concise and readable in many languages, either because functions are curried ( filter odd ), because they're methods of their sequence argument ( mylist.filter ) or because there's a convenient slicing ( (`filter` mylist) ) or partial application ( (_.filter(odd)) ) syntax. With python, there's none of those things, so I usually have to use lambdas (or even unnecessary named functions)...which means there's not really any advantage of functions over comprehensions.

Sometimes I do pass unapplied map or filter as an argument (usually to a fold function, or something similar). But this doesn't really work in python, either. Python only has a left fold and the argument order is the wrong way around for this sort of thing, so you have to flip it.

User avatar
Aaeriele
Posts: 2127
Joined: Tue Feb 23, 2010 3:30 am UTC
Location: San Francisco, CA

Re: Coding: Fleeting Thoughts

Postby Aaeriele » Thu Nov 21, 2013 6:25 pm UTC

Also, what's this about map et al not being first-class?

Code: Select all

def apply(func, args):
   return func(*args)
   
print apply(map, [lambda x: x+5, [6, 7, 8, 9, 10]])

# prints [11, 12, 13, 14, 15]
Vaniver wrote:Harvard is a hedge fund that runs the most prestigious dating agency in the world, and incidentally employs famous scientists to do research.

afuzzyduck wrote:ITS MEANT TO BE FLUTTERSHY BUT I JUST SEE AAERIELE! CURSE YOU FORA!

troyp
Posts: 557
Joined: Thu May 22, 2008 9:20 pm UTC
Location: Lismore, NSW

Re: Coding: Fleeting Thoughts

Postby troyp » Thu Nov 21, 2013 6:50 pm UTC

Do you mean me? I said comprehensions aren't first-class (as opposed to map and filter, which are). Eg. I couldn't pass an "unapplied" comprehension as an argument, to be later "applied" to a list (I'd have to explicitly use a lambda for that effect).

aside: The fora are fixed, hurray!


Return to “Coding”

Who is online

Users browsing this forum: Exabot [Bot] and 8 guests