Coding: Fleeting Thoughts

A place to discuss the implementation and style of computer programs.

Moderators: phlip, Moderators General, Prelates

User avatar
Link
Posts: 1324
Joined: Sat Mar 07, 2009 11:33 am UTC
Location: ᘝᓄᘈᖉᐣ
Contact:

Re: Coding: Fleeting Thoughts

Postby Link » Sat Mar 28, 2015 10:47 pm UTC

FT: finding a good design pattern for a complex library can be difficult. Right now, I'm writing a Python package where I want to have a single function in one module, resource.py, that can load various kinds of resources. However, some of the resources in turn need to load other resources. All nice and dandy if I keep everything in one file, but I'd like to have the individual resource classes in their own files. The problem is that this gives circular dependencies. So basically, I'd like to have a layout like this:

Code: Select all

my_package/
    resource.py
    resource_a.py
    resource_b.py
    resource_c.py
resource.py can load all the resource types and therefore uses from . import resource_a, resource_b, resource_c. Resource type A needs to load other resources using the general loading mechanism and therefore needs from . import resource.py. Resource types B and C can be used on their own. Putting the code from resource_a.py in resource.py would fix the circular dependency, but would lead to a huge file and would be semantically ugly. Alternatively, I could just make it illegal to import resource_a directly, but if I then keep it legal to import resources B and C directly, that would again be semantically ugly. The fix to that would be to make it illegal to import any of the resource types directly, but that would to excessive amounts of imported code if only resources B or C are needed. Ideally, I'd like to have the ability to make it possible to import any module from the package on its own, such that no more code is pulled in than is strictly necessary. Problem is, I have no idea how to do that without fragile, ugly hacks.

User avatar
Yakk
Poster with most posts but no title.
Posts: 11045
Joined: Sat Jan 27, 2007 7:27 pm UTC
Location: E pur si muove

Re: Coding: Fleeting Thoughts

Postby Yakk » Sun Mar 29, 2015 1:56 am UTC

Use a header file pattern. Well, sort of.

resource_a_h.py presumes you are resource.py and are importing it.

resource_a.py does not. It is externally loadable, and it imports resource.py. (and resource_a_h.py).

Or somesuch.

(I don't know pythonic import well enough to ensure this would work, or be clean).
One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision - BR

Last edited by JHVH on Fri Oct 23, 4004 BCE 6:17 pm, edited 6 times in total.

User avatar
Link
Posts: 1324
Joined: Sat Mar 07, 2009 11:33 am UTC
Location: ᘝᓄᘈᖉᐣ
Contact:

Re: Coding: Fleeting Thoughts

Postby Link » Tue Mar 31, 2015 5:13 am UTC

That would probably work, but it's not quite a Pythonic way to do things (i.e. it's an ugly hack). It would be better if I could change the design pattern to something more linear. Right now I'm thinking about a pattern by which I give each resource type its own loader (right now they have simple constructors, and resource.py massages external data into a format the resources can work with), and giving resource.py some code that just determines the type of data I'm dealing with. That would lead to some boilerplate code to actually act on the answer, but it would be cleaner.

Ubik
Posts: 1016
Joined: Thu Oct 18, 2007 3:43 pm UTC

Re: Coding: Fleeting Thoughts

Postby Ubik » Tue Mar 31, 2015 6:23 am UTC

I looked up python circular imports, and apparently they should work if the other module just isn't used during module load time. I think.

User avatar
Link
Posts: 1324
Joined: Sat Mar 07, 2009 11:33 am UTC
Location: ᘝᓄᘈᖉᐣ
Contact:

Re: Coding: Fleeting Thoughts

Postby Link » Tue Mar 31, 2015 6:28 am UTC

s
Ubik wrote:I looked up python circular imports, and apparently they should work if the other module just isn't used during module load time. I think.

Not if you use relative imports, i.e. 'from . import module' -- which is generally good practice to do. (That may be different for Python 2 [I don't know], but I'm using Python 3.)

User avatar
karhell
Posts: 656
Joined: Wed Jun 19, 2013 4:56 pm UTC
Location: Breizh

Re: Coding: Fleeting Thoughts

Postby karhell » Tue Mar 31, 2015 7:04 am UTC

This might be absolutely silly, but how about simply outsourcing the contents from resource.py that resource_a.py needs to resource_d.py ?
AluisioASG wrote:191 years ago, the great D. Pedro I drew his sword and said: "Indent thy code or die!"
lmjb1964 wrote:We're weird but it's okay.
iskinner, thunk, GnomeAnne, Quantized, and any other Blitzers, have fun on your journey!
Wait well, ChronosDragon, may your seaish ketchup be molpish!

User avatar
phlip
Restorer of Worlds
Posts: 7538
Joined: Sat Sep 23, 2006 3:56 am UTC
Location: Australia
Contact:

Re: Coding: Fleeting Thoughts

Postby phlip » Tue Mar 31, 2015 1:21 pm UTC

Ubik wrote:I looked up python circular imports, and apparently they should work if the other module just isn't used during module load time. I think.

Basically, the problem is: say modules A and B both depend on each other. Your program imports module A. Right at the top (presumably), A imports module B. Module B then re-imports module A... but because module A is already created, it just gets the existing module object back. But module A hasn't actually run yet, so that module object is empty.

In theory, B then runs through, and all it does is create a bunch of functions, maybe a few classes, and finishes. Then A runs through, and does the same. Then by the time you actually call any of those functions in either module (or create objects of any of those classes), both are fully loaded, and anything in either module that calls the other will work fine. (In Python a function can reference variables and functions and stuff that don't exist yet, as long as they exist by the time the function is called.)

However, it means that if there's anything in the mainline of B that references stuff in A, it will fail, as the stuff just isn't there yet. The main causes are using "from A import stuff" or subclassing classes from A in B, but it can also be calling init functions in A from the B mainline, trying to re-publish objects from A by just running "foo = A.foo" in the B mainline, or similar things.
However, if the import is just "import A", and the only place A is used is inside functions that aren't called immediately, so B can finish loading without referring to anything from A, then A can finish loading, and then after that everything should work OKish. But it's pretty fragile... it won't take much for it to suddenly start failing in weird ways again. And often ends up depending on which one's imported first (if you write "import A" first in your mainline then B won't be able to see A at loadtime, but if you write "import B" first then it'll be vice-versa).

There's some simple things you can do to fix it, but most of them require already being in that "if you just use import A instead of from-import you'll be fine" state, and rearranging it so it's less fragile:
  • In one of A or B, don't import the other at the main line, move the imports into the functions it's actually needed for. This pretty much requires that one of the two is don't that "don't require the other at the mainline" thing, and is only requiring the other inside the functions. This usually means picking one of the two modules to be "more basic"... like, if I have a "config" module for handling global config and a "utils" module, and both rely on each other (the config loader relies on some utils, while other utils rely on the config setup) then usually the utils module will be considered "more basic", so the config module will import the utils module at the main line, but the utils module will only import the config module inside the different util functions that require it.
  • Move the imports from the top to the bottom of the modules, so that each module is loaded before it tried to load the other. Again, this requires that nothing in the mainline in the modules require the other module to be there.
  • Put the imports in the middle of the module, so that eg module A goes: "<stuff that B depends on> import B <stuff that depends on B>"... probably a bad idea, chances are if you require this then you're doing something very wrong anyway.
  • Break the modules up into more modules to try to break the circular dependency (if the stuff in B that depends on A and the stuff in B that A depends on are different things, like in the "utils module" example, then B could possibly be split into two to break the loop).

Code: Select all

enum ಠ_ಠ {°□°╰=1, °Д°╰, ಠ益ಠ╰};
void ┻━┻︵​╰(ಠ_ಠ ⚠) {exit((int)⚠);}
[he/him/his]

User avatar
Link
Posts: 1324
Joined: Sat Mar 07, 2009 11:33 am UTC
Location: ᘝᓄᘈᖉᐣ
Contact:

Re: Coding: Fleeting Thoughts

Postby Link » Tue Mar 31, 2015 4:13 pm UTC

karhell wrote:This might be absolutely silly, but how about simply outsourcing the contents from resource.py that resource_a.py needs to resource_d.py ?

That's not an option. resource_a.py needs the resource loader defined in resource.py, and the resource loader needs resource_a.py because it can load resources of type A.

phlip wrote:In one of A or B, don't import the other at the main line, move the imports into the functions it's actually needed for. This pretty much requires that one of the two is don't that "don't require the other at the mainline" thing, and is only requiring the other inside the functions. This usually means picking one of the two modules to be "more basic"... like, if I have a "config" module for handling global config and a "utils" module, and both rely on each other (the config loader relies on some utils, while other utils rely on the config setup) then usually the utils module will be considered "more basic", so the config module will import the utils module at the main line, but the utils module will only import the config module inside the different util functions that require it.
That might actually work. It's not the prettiest of solutions, but it might still be the best one.

bittyx
Posts: 194
Joined: Tue Sep 25, 2007 9:10 pm UTC
Location: Belgrade, Serbia

Re: Coding: Fleeting Thoughts

Postby bittyx » Tue Mar 31, 2015 5:53 pm UTC

Link wrote:
karhell wrote:This might be absolutely silly, but how about simply outsourcing the contents from resource.py that resource_a.py needs to resource_d.py ?

That's not an option. resource_a.py needs the resource loader defined in resource.py, and the resource loader needs resource_a.py because it can load resources of type A.

phlip wrote:In one of A or B, don't import the other at the main line, move the imports into the functions it's actually needed for. This pretty much requires that one of the two is don't that "don't require the other at the mainline" thing, and is only requiring the other inside the functions. This usually means picking one of the two modules to be "more basic"... like, if I have a "config" module for handling global config and a "utils" module, and both rely on each other (the config loader relies on some utils, while other utils rely on the config setup) then usually the utils module will be considered "more basic", so the config module will import the utils module at the main line, but the utils module will only import the config module inside the different util functions that require it.
That might actually work. It's not the prettiest of solutions, but it might still be the best one.


Disclaimer: I've only ever used Python for playing with Project Euler problems and small scripts, so I have no idea how imports work, nor how to write idiomatic code, nor how to write Python modules for that matter.

Is it possible to use some form of dependency injection? E.g. in each of the resources' constructors, have one argument be a reference to the resource loader itself, so when the resource loader creates a resource, it injects itself through the resource's constructor, thereby giving the resource a reference to the loader. That way, none of the resources would have to instantiate the resource loader, so they wouldn't even have to import it. Or does this make no sense in Python?

Python-ish pseudo-code sort-of thing follows (please don't murder me, Python programmers). So assuming you have something like this:

resource_loader.py:

Code: Select all

from . import resource_foo.py
from . import resource_bar.py

class ResourceLoader:
    def loadResource(type):
        switch (type):
            case "foo":
                return new ResourceFoo
            case "bar":
                return new ResourceBar
            default:
                throw new Exception("whatever")


resource_foo.py:

Code: Select all

from . import resource_loader.py

class ResourceFoo:
    def doStuff():
        return 42

    def doMoreStuff():
        loader = new ResourceLoader
        bar = loader.load("bar")
        return bar.doBarStuff()


resource_bar.py:

Code: Select all

from . import resource_loader.py

class ResourceBar:
    def doBarStuff():
        return 13

    def runOutOfIdeas():
        loader = new ResourceLoader
        foo = loader.load("foo")
        return foo.doStuff()


You could refactor it into something like this instead:

resource_loader.py:

Code: Select all

from . import resource_foo.py
from . import resource_bar.py

class ResourceLoader:
    def loadResource(type):
        switch (type):
            case "foo":
                return new ResourceFoo(self)
            case "bar":
                return new ResourceBar(self)
            default:
                throw new Exception("whatever")


resource_foo.py:

Code: Select all

class ResourceFoo:
    def __init__(loader):
        self.loader = loader

    def doStuff():
        return 42

    def doMoreStuff():
        bar = self.loader.load("bar")
        return bar.doBarStuff()


resource_bar.py:

Code: Select all

class ResourceBar:
    def __init__(loader):
        self.loader = loader

    def doBarStuff():
        return 13

    def runOutOfIdeas():
        foo = self.loader.load("foo")
        return foo.doStuff()


This way neither the resource_foo nor resource_bar classes need to import the resource_loader, as it is handed to them at instantiation, so you avoid circular imports. Is this something you can do in Python? Quick Googling shows that you *can* do dependency injection, but I have no idea if this is applicable to your use-case.

Sorry if this was absolutely unhelpful.

troyp
Posts: 557
Joined: Thu May 22, 2008 9:20 pm UTC
Location: Lismore, NSW

Re: Coding: Fleeting Thoughts

Postby troyp » Wed Apr 01, 2015 8:47 am UTC

Python modules are just objects, so you should be able to do whatever you want with them, dynamically. I'm not sure how portable it would be though.

Basically, you can use the imp module to create a module, place members into its __dict__ directly and then and shove it into sys.modules to make it available for import. So if you wanted, you could leave everything in one file, which would export core functionality and add several modules to sys.modules, and the user can then import whichever extra modules they want. Or split the code into separate files and use execfile from the main module to load the definitions into the module __dict__s.

Code: Select all

# resource.py

import imp as _imp
import sys as _sys

_resource_a = _imp.new_module("resource_a")
execfile("resource_a.py", _resource_a.__dict__)
_sys.modules["resource_a"] = _resource_a

Code: Select all

# resource_a.py

def foo():
    return "foo"
bar = 4
class baz():
    pass

Of course, you'd only want to resort to that if you couldn't get things to work out with statically defined modules.

User avatar
Link
Posts: 1324
Joined: Sat Mar 07, 2009 11:33 am UTC
Location: ᘝᓄᘈᖉᐣ
Contact:

Re: Coding: Fleeting Thoughts

Postby Link » Wed Apr 01, 2015 4:48 pm UTC

Link wrote:
phlip wrote:In one of A or B, don't import the other at the main line, move the imports into the functions it's actually needed for. [...]
That might actually work. It's not the prettiest of solutions, but it might still be the best one.
Turns out that actually works just fine. :)

bittyx: that's possible, but adds more complexity than I want.
troyp: frankly, if you need code like that, that's probably a sign you're doing it wrong. ;)

troyp
Posts: 557
Joined: Thu May 22, 2008 9:20 pm UTC
Location: Lismore, NSW

Re: Coding: Fleeting Thoughts

Postby troyp » Thu Apr 02, 2015 10:54 am UTC

Link wrote:troyp: frankly, if you need code like that, that's probably a sign you're doing it wrong. ;)

I don't think that's true at all.

Surely you realize static imports are extremely limited. We often don't notice it, since we tend to focus on the solutions that are available given the tools we have, but there's a whole range of possibilities that are excluded if you rule out dynamic modules. Even something as simple as parametrizing a module isn't possible with Python's static modules. Modules are Python's primary means of namespace management, but with static imports, you're limited to just importing names that were defined in advance. Kinda lame for a dynamic language. Do you really think every possibility outside of that tiny sliver of the design space is "doing it wrong"?

Certainly, there are some (fairly generic) reason's to avoid dynamically creating modules if you don't really need them:
  • Dynamic code shouldn't be used if it's not needed;
  • Simple solutions are better than complex solutions, other things being equal; and
  • Idiomatic solutions are better than unusual solutions, other things being equal.
There's also the issue that the greater flexibility of dynamically created modules could be used to work around issues caused by a flawed design. Again, that's a pretty generic concern that applies whenever an "advanced" (more complex, obscure or in some way costly) feature is used when it shouldn't really be necessary.

Those things aside, though, there's nothing inherently wrong with a design that requires dynamic modules. I'm not saying you should use them: you probably don't need them. But if it turns out there's something about a problem that's running into limitations of Python's static modules, and you can only hack around it awkwardly, then it's worth considering dynamically-generated modules.

edit:
Link wrote:
Link wrote:
phlip wrote:In one of A or B, don't import the other at the main line, move the imports into the functions it's actually needed for. [...]
That might actually work. It's not the prettiest of solutions, but it might still be the best one.
Turns out that actually works just fine. :)

Btw, to disagree again, there's nothing ugly at all about using import in a function. You're just bringing in function-local names instead of "global" (module-level) ones. If anything, it's better. If you don't need the names in the rest of the module, why put them in the global namespace? *

Unless of course, you have to import them into a bunch of functions in the same module: I agree it becomes inelegant at that point.

* Well, one reason is to know about import failure at module-load time (also, just to have them grouped together). Admittedly, I do tend to put mine at the top even if I just use a single imported name in one function... but only if that import is necessary for the "core functionality" of the module. If it's something "extra" (peripheral to the purpose of the module - eg. handling some unusual file type), I'm happy to move it into the function (in fact, the arguments I gave before tend to turn on their head in this case: I may not want the "extra" import grouped with the "important" ones and I may not want an error unless the functionality is actually invoked). If I had another reason to put import in a function, like preventing circular imports, I'd consider that a good reason too. I wouldn't (necessarily) consider it hackish; really, putting them at the top is what's hackish.

nb. I've been abusing terminology pretty badly in the above post. Python doesn't have static modules, of course. They're always dynamic. It's just that the standard way of doing things, defining module files and importing them, doesn't make use of their dynamic nature.

User avatar
Link
Posts: 1324
Joined: Sat Mar 07, 2009 11:33 am UTC
Location: ᘝᓄᘈᖉᐣ
Contact:

Re: Coding: Fleeting Thoughts

Postby Link » Thu Apr 02, 2015 8:57 pm UTC

troyp wrote:Those things aside, though, there's nothing inherently wrong with a design that requires dynamic modules. I'm not saying you should use them: you probably don't need them. But if it turns out there's something about a problem that's running into limitations of Python's static modules, and you can only hack around it awkwardly, then it's worth considering dynamically-generated modules.
OK, I suppose I agree with that, but I think when you get to the point where that becomes the best option, you should probably also take a step back to consider the design choices you made that led to that point. In any case, a few circular imports here and there almost certainly aren't, by themselves, enough cause to leap into dynamic modules.

troyp wrote:Btw, to disagree again, there's nothing ugly at all about using import in a function. You're just bringing in function-local names instead of "global" (module-level) ones. If anything, it's better. If you don't need the names in the rest of the module, why put them in the global namespace? *

Unless of course, you have to import them into a bunch of functions in the same module: I agree it becomes inelegant at that point.

* Well, one reason is to know about import failure at module-load time (also, just to have them grouped together). Admittedly, I do tend to put mine at the top even if I just use a single imported name in one function... but only if that import is necessary for the "core functionality" of the module. If it's something "extra" (peripheral to the purpose of the module - eg. handling some unusual file type), I'm happy to move it into the function (in fact, the arguments I gave before tend to turn on their head in this case: I may not want the "extra" import grouped with the "important" ones and I may not want an error unless the functionality is actually invoked). If I had another reason to put import in a function, like preventing circular imports, I'd consider that a good reason too. I wouldn't (necessarily) consider it hackish; really, putting them at the top is what's hackish.
Importing a module is of a side-effect than I generally like my functions to have. It's all nice and dandy when the module you're importing is just a few lines, but consider the case where you import a module that has a long and complicated dependency chain of its own: that's not something I want to happen behind the scenes when calling some arbitrary function. Special init-functions whose entire purpose is to get everything ready for use are, of course, an exception.

User avatar
felltir
has a sniper scope and a trigger finger.
Posts: 2491
Joined: Tue Mar 04, 2008 5:01 pm UTC
Location: Back in't home town. Never at home.
Contact:

Re: Coding: Fleeting Thoughts

Postby felltir » Fri Apr 03, 2015 12:52 pm UTC

Dependancy resolution is hard.
Spoiler:
RoadieRich wrote:He's a super flexible furry martial artist from London. She is a Rabbit breeding mad scientist from Michigan. They fight crime!
The Great Hippo wrote:I THINK THE SOLAR SYSTEM MIGHT BE AN ATOM OF OXYGEN.


Blog

he/him/his

bittyx
Posts: 194
Joined: Tue Sep 25, 2007 9:10 pm UTC
Location: Belgrade, Serbia

Re: Coding: Fleeting Thoughts

Postby bittyx » Fri Apr 03, 2015 10:34 pm UTC

Circular dependencies are not inherently a problem, and sometimes they are required, and are actually the correct solution - but more often than not, they can be a sign of bad design, at least in my experience (obviously this should be evaluated in each case). In the latter situation, you can look for hidden dependencies and extract them into a separate class (or module or whatever). So, I'm not saying that you're doing anything wrong, but if you were more specific about what you're trying to accomplish, maybe we could offer better advice, like a completely different approach (beware of the XY problem).

Note: The first article linked talks about constructor dependency injection in particular, but the logic might be applicable to circular dependencies in general.

troyp
Posts: 557
Joined: Thu May 22, 2008 9:20 pm UTC
Location: Lismore, NSW

Re: Coding: Fleeting Thoughts

Postby troyp » Sat Apr 04, 2015 6:06 am UTC

Link wrote:OK, I suppose I agree with that, but I think when you get to the point where that becomes the best option, you should probably also take a step back to consider the design choices you made that led to that point. In any case, a few circular imports here and there almost certainly aren't, by themselves, enough cause to leap into dynamic modules.

Yeah, I agree with that. "Static" modules are fine for the common cases, including common cases of circular imports, and if you're thinking about using dynamic modules, it's a good time to consider whether you really need them or whether you can simplify the design to remove the problem.

Importing a module is of a side-effect than I generally like my functions to have. It's all nice and dandy when the module you're importing is just a few lines, but consider the case where you import a module that has a long and complicated dependency chain of its own: that's not something I want to happen behind the scenes when calling some arbitrary function. Special init-functions whose entire purpose is to get everything ready for use are, of course, an exception.

Hmmm... are there really likely to be side-effects to worry about? Apart from the possibility of ImportErrors (which you could handle in the function if appropriate), most modules should only export names - and those are brought into the current scope (so they become local variables in a function and class attributes in a class).

Actually, there is one other major side-effect that occurs with any import. Although the scoping itself should be nice and neat, imports are tricky since the actual creation of a module object from the module file only seems to occur once. The first time you import a module, its contents are "fixed", so even if the contents change, reimporting won't bring in new names. That's well known at top level, but it even applies when you reimport a module in other scopes. The creation of a module object from a file seems to be a one-time global event. My impression is that sys.modules is used as a register of modules: ie. when a module file is first read, a module object is created and maintained by a binding in sys.modules; then, when the module is imported again - anywhere - sys.modules is checked first, before the runtime looks for a module file to compile and load. That could definitely be an issue if you have some dynamic module that changes at runtime and want to use it at function level in a dynamic way.1 Still, that's a fairly obscure use case.2 As long as the module doesn't change, the exact time of first import shouldn't matter, should it? Or am I missing something?

[1] Actually, even if the module isn't changing after creation, it could still be a problem, since sys.modules retrieves modules based on name (rather than object id). So if you have some sort of module factory that generates multiple instantiations of a module, and you want to import different instances in different functions, that's probably not going to work if the instances all have the same name. You'd need to introduce gensym functionality to ensure that each instance has a unique name (well, I guess if you were only importing a given instance once, you could just overwrite the sys.modules entry each time, but that sounds like a horrible solution).
[2] You should be able to permit multiple versions of a module by using reload() or even just deleting a module's entry in sys.modules. I'm not entirely certain about that, though; I've never tried this. Also, that wouldn't cause new versions of that module's dependencies to be created, so if that were required you'd have to do something else (I'm not sure if there's a way to do this provided or if you'd have to write the functionality yourself).

bittyx wrote:Circular dependencies are not inherently a problem, and sometimes they are required, and are actually the correct solution - but more often than not, they can be a sign of bad design, at least in my experience (obviously this should be evaluated in each case). In the latter situation, you can look for hidden dependencies and extract them into a separate class (or module or whatever)..

I think part of the reason people end up with dependency issues like this is that there are conflicting reasons for collecting definitions into modules. Suppose:
  • B and C call A
  • C calls B
Then:
  • to handle the dependencies, we might want to separate those definitions into 3 modules with dependencies A <- B <- C.
  • However, when we want to group the definitions for the sake of a human reader's comprehension and/or for the sake of namespace management, we may prefer to group them {B} {A,C}
If we concentrate on the latter two considerations when designing our modules... presto, circular dependencies.

I think a good way of reconciling the various motivations is to use a layered approach.
  • We split A into:
    • a low level component _A (imported by other "library" modules); and
    • a high level component A' (implemented using _A) imported by the "application" module(s).
  • We make B depend on _A instead of A, and use the grouping {_A} {B} {A',C}

User avatar
Thesh
Made to Fuck Dinosaurs
Posts: 5341
Joined: Tue Jan 12, 2010 1:55 am UTC
Location: Colorado

Re: Coding: Fleeting Thoughts

Postby Thesh » Sun Apr 05, 2015 5:18 am UTC

You know, if you are trying to find the multiplicative inverse modulo 2^n, I think the extended euclidean algorithm is just too much work.

Code: Select all

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char **argv)
{
   if (argc == 2)
   {
      unsigned long x = strtoul(argv[1], NULL, 10);
      if (x % 2 == 1)
      {
         unsigned long y = 1, z = x, i=0;
         while (z > 1)
         {
            unsigned long k = z & (1ul << ++i);
            y |= k;
            z += x*k;
         }
         printf("%lu\n",y);
      }
   }
   return 0;
}
Honesty replaced by greed, they gave us the reason to fight and bleed
They try to torch our faith and hope, spit at our presence and detest our goals

User avatar
felltir
has a sniper scope and a trigger finger.
Posts: 2491
Joined: Tue Mar 04, 2008 5:01 pm UTC
Location: Back in't home town. Never at home.
Contact:

Re: technomancy: Fleeting Thoughts

Postby felltir » Sun Apr 05, 2015 8:59 pm UTC

Also, I don't think I posted this here. I did a talk at the London Ruby User Group. It's about the weird bits of ruby, and it's ~10 minutes. Here it is.

Any feedback welcome!
Spoiler:
RoadieRich wrote:He's a super flexible furry martial artist from London. She is a Rabbit breeding mad scientist from Michigan. They fight crime!
The Great Hippo wrote:I THINK THE SOLAR SYSTEM MIGHT BE AN ATOM OF OXYGEN.


Blog

he/him/his

User avatar
hotaru
Posts: 1017
Joined: Fri Apr 13, 2007 6:54 pm UTC

Re: technomancy: Fleeting Thoughts

Postby hotaru » Sun Apr 05, 2015 10:41 pm UTC

Thesh wrote:You know, if you are trying to find the multiplicative inverse modulo 2^n, I think the extended euclidean algorithm is just too much work.

surely this would be less work than all that bit twiddling?

Code: Select all

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char *argv[argc])
{ if(argc == 2)
  { unsigned long x = strtoul(argv[1], NULL, 10);
    if(x % 2)
    { unsigned long y = 1;
      for(unsigned long z = x; z > 1; z *= z) y *= z;
      printf("%lu\n", y); }}
  return 0; }

Code: Select all

factorial product enumFromTo 1
isPrime n 
factorial (1) `mod== 1

User avatar
Thesh
Made to Fuck Dinosaurs
Posts: 5341
Joined: Tue Jan 12, 2010 1:55 am UTC
Location: Colorado

Re: technomancy: Fleeting Thoughtth

Postby Thesh » Mon Apr 06, 2015 12:18 am UTC

Well, depends on whether you want to make your code as difficult to read as possible.
Honesty replaced by greed, they gave us the reason to fight and bleed
They try to torch our faith and hope, spit at our presence and detest our goals

User avatar
Xeio
Friends, Faidites, Countrymen
Posts: 5080
Joined: Wed Jul 25, 2007 11:12 am UTC
Location: C:\Users\Xeio\
Contact:

Re: technomancy: Fleeting Thoughtth

Postby Xeio » Mon Apr 06, 2015 4:20 am UTC

felltir wrote:Also, I don't think I posted this here. I did a talk at the London Ruby User Group. It's about the weird bits of ruby, and it's ~10 minutes. Here it is.

Any feedback welcome!
So a bit like "wat" specific to Ruby?

The ambiguity in method calls for Ruby was something that always bothered me, you don't really know what "a b c" is by looking at it unless you know exactly what a and b are (at runtime no less). It could be one method call with two parameters, or it could be two calls with one parameter each.

EvanED
Posts: 4324
Joined: Mon Aug 07, 2006 6:28 am UTC
Location: Madison, WI
Contact:

Re: technomancy: Fleeting Thoughtth

Postby EvanED » Mon Apr 06, 2015 4:38 am UTC

felltir wrote:Also, I don't think I posted thith here. I did a talk at the London Ruby User Group. It's about the weird bits of ruby, and it's ~10 minutes. Here it is.

Any feedback welcome!
Just took a look. FWIW, I use the "adjacent strings get concatenated" thing in C++ with some frequency, because it provides a way to break up a long string constant onto multiple lines. (It also kinda shows up in macros with the ## operation, but that's not quite the same thing.)

Xeio wrote:So a bit like "wat" specific to Ruby?
With a lot less headdesk and evidence of awful language "design" anyway. (Only knowing some things about ruby, the regex thing seems like a definite headdesk, but nothing else seems terribly egregious. "2".to_i +2 comes close, but note that both "2".to_i+2 and "2".to_i + 2 evaluate to 4 according to repl.it.)

User avatar
Dinosaur!
Posts: 30
Joined: Fri Dec 14, 2012 8:45 pm UTC
Location: Santa Cruz, CA

Re: ant farming: Fleeting Thoughtth

Postby Dinosaur! » Mon Apr 06, 2015 6:43 am UTC

@felltir:

Spoiler:

Code: Select all

?????:%%%

gave me a tickle

User avatar
hotaru
Posts: 1017
Joined: Fri Apr 13, 2007 6:54 pm UTC

Re: technomancy: Fleeting Thoughtth

Postby hotaru » Mon Apr 06, 2015 6:40 pm UTC

Thesh wrote:Well, depends on whether you want to make your code as difficult to read as possible.

i suppose, but there are tools that can automate that.
it's also possible to do it with just one (vector) multiply per loop iteration:

Code: Select all

#include <limits.h>
#include <stdio.h>
#include <stdlib.h>

#if ULONG_MAX == 0xffffffffffffffffull
#define VECTOR_SIZE 16
#elif ULONG_MAX == 0xffffffffull
#define VECTOR_SIZE 8
#endif

int main(int argc, char *argv[argc])
{ for(int i = 1; i < argc; ++i)
  { unsigned long x = strtoul(argv[i], NULL, 10);
    if(x & 1)
    { unsigned long y __attribute__ ((vector_size (VECTOR_SIZE))) = {1, x};
      while(y[1] & ~1ul) y *= y[1];
      printf("%lu: %lu\n", x, y[0]); }
    else printf("%lu: not invertible\n", x); }
  return 0; }

Code: Select all

factorial product enumFromTo 1
isPrime n 
factorial (1) `mod== 1

User avatar
Xanthir
My HERO!!!
Posts: 5186
Joined: Tue Feb 20, 2007 12:49 am UTC
Location: The Googleplex
Contact:

Re: Coding: Fleeting Thoughts

Postby Xanthir » Thu Apr 09, 2015 4:01 pm UTC

troyp wrote:I didn't realize UX experts were ever sober...

Is anyone familiar with the "separator" characters from ASCII (you know, file separator, group separator, etc)? And the other control codes, for that matter?

[snip]

I've often wondered about these codes: how often are they encountered these days? Are they "available" to be used as special characters in a data format containing text, to avoid escaping?

The entire non-printable ASCII range 0-1F isn't used these days, besides null, tab, and the newlines. They're a variety of oddities that probably made sense back in the day when every byte mattered in transmissions, but are completely irrelevant now.

If you wanted to make a data format that was hard to read and write, sure, you can use the separator chars to separate records/lines/etc. But using printable chars and escaping is a better idea.
(defun fibs (n &optional (a 1) (b 1)) (take n (unfold '+ a b)))

Ubik
Posts: 1016
Joined: Thu Oct 18, 2007 3:43 pm UTC

Re: Coding: Fleeting Thoughts

Postby Ubik » Thu Apr 09, 2015 6:58 pm UTC

void * warranty = NULL;
*warranty;

User avatar
Yakk
Poster with most posts but no title.
Posts: 11045
Joined: Sat Jan 27, 2007 7:27 pm UTC
Location: E pur si muove

Re: Coding: Fleeting Thoughts

Postby Yakk » Mon Apr 13, 2015 2:39 am UTC

So, rather than perfect forwarding in a method, what if we type erased down to assignment to T?

We get nearly the efficiency of perfect forwarding, with far less code generation (if the function is expensive), the ability to have a virtual interface, and the perfect forwarding mumbo-jumbo and SFINAE tests can be done in a utility class instead of in "client code".
One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision - BR

Last edited by JHVH on Fri Oct 23, 4004 BCE 6:17 pm, edited 6 times in total.

User avatar
You, sir, name?
Posts: 6971
Joined: Sun Apr 22, 2007 10:07 am UTC
Location: Chako Paul City
Contact:

Re: Coding: Fleeting Thoughts

Postby You, sir, name? » Thu Apr 16, 2015 7:21 pm UTC

Ubik wrote:void * warranty = NULL;
*warranty;


rimshot
I edit my posts a lot and sometimes the words wrong order words appear in sentences get messed up.

Ubik
Posts: 1016
Joined: Thu Oct 18, 2007 3:43 pm UTC

Re: Coding: Fleeting Thoughts

Postby Ubik » Fri Apr 17, 2015 7:29 pm UTC

Agh. I recently made a pull request on GitHub and it got accepted too. Yay. The thing is, I've been using my work laptop lately on free time too because I haven't really been able to use my personal machine. Of course that commit has my work email in it now. It's not that big deal, especially because apparently it isn't even googleable, but still would have been nicer to keep the line between private and work coding more clear.

As a temporary "fix", I just set my email in .gitconfig to "(none)", so at least I get to set it explicitly for every cloned repo. I don't work with so many that setting it for all of them is impractical. Still, it'd be sort of nice if I could overwrite a setting for an entire directory at once. Without some hook trickery, preferably, though maybe I should figure out how they're used anyway. Still, agh.

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6793
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: Coding: Fleeting Thoughts

Postby The Great Hippo » Mon Apr 20, 2015 10:11 am UTC

I'm having issues with a really trivial problem; I suspect I'm over-thinking it.

I'm dealing with coordinates in Python; 2D, 3D, and 4D. The 2D coordinates are used for GUI stuff; the 3D coordinates are used for game-space stuff, and the 4D coordinates are a special case for describing walls in game-space (the fourth slot is used for another 3D vector that points in the direction the wall is facing).

I'm having a brainfart over how to arrange my coordinate objects. Should I use separate vector and point objects for each? A 2D Vector object, a 2D Point object, a 3D Vector object, a 3D Point object...? Or should I generalize? A Vector and Point object that takes an arbitrary number of arguments, but throws an exception if you try to use them with another Vector/Point that didn't take the same number of arguments?

Part of me is tempted to just make them both the same thing; a Coordinate object that inherits from lists (or maybe tuples, since I think they should be immutable) that allows for addition and subtraction (and returns a new Coordinate object when you do). Should I even bother differentiating between points and vectors? I know the basic difference ("I'll see you in five hours" -- Vector; "I'll see you at 3 o'clock on January 2nd" -- Point), but when it comes to the code I'm writing, I suspect the difference may largely be academic -- not functional. Am I wrong? Is it generally 'good practice' to keep points and vectors separate (and not allow -- for example -- addition of points)?

I might be doing some semi-complicated stuff with the 3D vectors, but -- at the moment -- I highly doubt it will involve matrices.

User avatar
Xenomortis
Not actually a special flower.
Posts: 1389
Joined: Thu Oct 11, 2012 8:47 am UTC

Re: Coding: Fleeting Thoughts

Postby Xenomortis » Mon Apr 20, 2015 11:09 am UTC

What would you gain by separating out the two concepts (vector vs point)?
For one thing, mathematically there is no difference.
Image

Ubik
Posts: 1016
Joined: Thu Oct 18, 2007 3:43 pm UTC

Re: Coding: Fleeting Thoughts

Postby Ubik » Mon Apr 20, 2015 11:20 am UTC

In a dynamic language like Python it might not make sense to build two largely similary types - though I think I read optional type annotations could be coming to Python? The 4D vector part sounds a bit suspicious to me, like something that could be better represented by an object having position and direction as separate members.

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6793
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: Coding: Fleeting Thoughts

Postby The Great Hippo » Mon Apr 20, 2015 11:28 am UTC

The only benefit I can think of is that there are conditions where it makes things more clear, and possibly has some function for raising exceptions. For example, a circle's center point is -- well, a point. Its radius can be described as a vector. If I ever confuse the two, knowing that one is a vector (and therefore cannot tell me anything meaningful about the shape's location) can be useful -- but possibly only in a really trivial way.

I'm sure there are some cases where it might be useful to raise an exception because I've mistaken a vector for a point (or vice versa), but I'm hard-pressed to think of any -- and that's a very peculiar mistake to make. I can't think of under what conditions I'd mistake a circle's radius for its location.
Ubik wrote:In a dynamic language like Python it might not make sense to build two largely similary types - though I think I read optional type annotations could be coming to Python? The 4D vector part sounds a bit suspicious to me, like something that could be better represented by an object having position and direction as separate members.
Yeah, I was actually thinking of that as I typed it out. It's probably really unwise to have vectors with elements that aren't all the same type (and even more unwise to have vectors using vectors as elements). It would make way more sense to just use a 3D coordinate, and slap the direction on as another (separate) 3D vector.

EDIT: Also, yeah, I remember reading last year that type annotations are very likely to become part of Python -- building on top of function annotations. You can already jury-rig type annotations using function annotations in Python 3.x.

User avatar
Yakk
Poster with most posts but no title.
Posts: 11045
Joined: Sat Jan 27, 2007 7:27 pm UTC
Location: E pur si muove

Re: Coding: Fleeting Thoughts

Postby Yakk » Mon Apr 20, 2015 9:57 pm UTC

Xenomortis wrote:What would you gain by separating out the two concepts (vector vs point)?
For one thing, mathematically there is no difference.

What? No.

Point+Point has no meaning -- points come from an affine set. alpha point + (1-alpha) point is a point.

Vector+Vector is a vector, as is scalar*vector.

Point+vector is a point. Point-point is a vector.

Zero is a distinguished vector, but there are no distinguished points.

Did you mean no arithmatic difference?
One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision - BR

Last edited by JHVH on Fri Oct 23, 4004 BCE 6:17 pm, edited 6 times in total.

User avatar
Dinosaur!
Posts: 30
Joined: Fri Dec 14, 2012 8:45 pm UTC
Location: Santa Cruz, CA

Re: Coding: Fleeting Thoughts

Postby Dinosaur! » Tue Apr 21, 2015 4:23 am UTC

It seems odd to me to specify a circle by a point and a vector. I mean, there's no reason that its radius cannot be described as a vector, but a scalar seems more natural to me.

User avatar
Xenomortis
Not actually a special flower.
Posts: 1389
Joined: Thu Oct 11, 2012 8:47 am UTC

Re: Coding: Fleeting Thoughts

Postby Xenomortis » Tue Apr 21, 2015 8:42 am UTC

Yakk wrote:Did you mean no arithmatic difference?

In En - his coordinate spaces.
(Granted, his four dimensional case may or may not work as a vector space).

Dinosaur! wrote:It seems odd to me to specify a circle by a point and a vector. I mean, there's no reason that its radius cannot be described as a vector, but a scalar seems more natural to me.

Maybe not a circle, but an ellipse would use vectors to specify the major and minor axis.
(Well I guess you could use one vector for the major/minor and a scalar for the minor/major, but that's just getting confusing).
Image

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6793
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: Coding: Fleeting Thoughts

Postby The Great Hippo » Tue Apr 21, 2015 10:36 am UTC

Dinosaur! wrote:It seems odd to me to specify a circle by a point and a vector. I mean, there's no reason that its radius cannot be described as a vector, but a scalar seems more natural to me.
Mostly, it's just 'cuz I was lazy and realized you can use a point and a vector to describe a lot of simple shapes -- so I created a general-purpose object that accepts a point and a vector and builds different shapes out of them (one of them being a circle).

...but yeah, it's probably a better idea to not be lazy and actually have the radius be a scalar rather than a vector. >.>
Xenomortis wrote:In En - his coordinate spaces.
(Granted, his four dimensional case may or may not work as a vector space).
The initial 4D space I was talking about is definitely not a vector space, since the fourth dimension was not composed of real numbers. But I've since come to my senses (well, 'relatively', at least) and realized that's nonsensical gabble-de-gook -- so I'm just going to stick to 2D and 3D vectors (no points, because I see no compelling reason why the origin should not be treated as a special and unique snowflake).

The big question I'm wrestling with then is whether or not 2D and 3D vectors should be handled by the same object. My intuition says no, because I see no cases where 2D and 3D vectors should be allowed to interact (and if they ever do, it should be handled by clearly stated methods, not built-in stuff).

User avatar
Xenomortis
Not actually a special flower.
Posts: 1389
Joined: Thu Oct 11, 2012 8:47 am UTC

Re: Coding: Fleeting Thoughts

Postby Xenomortis » Tue Apr 21, 2015 10:43 am UTC

Well just about the only difference between 2d and 3d vectors is that the latter can have a cross product operator.
I think I'd just have one class and assertions around all my operators making sure the vector dimensions match.
In C++ I might experiment with templated vector dimensions, but Python is a dynamic language.

The Great Hippo wrote: I created a general-purpose object that accepts a point and a vector and builds different shapes out of them (one of them being a circle).

...but yeah, it's probably a better idea to not be lazy and actually have the radius be a scalar rather than a vector. >.>

When choosing between two options, the inconsistent one is rarely better.
Image

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6793
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: Coding: Fleeting Thoughts

Postby The Great Hippo » Tue Apr 21, 2015 11:04 am UTC

Xenomortis wrote:Well just about the only difference between 2d and 3d vectors is that the latter can have a cross product operator.
I think I'd just have one class and assertions around all my operators making sure the vector dimensions match.
In C++ I might experiment with templated vector dimensions, but Python is a dynamic language.

Fair enough; I do love consistency and it's not *hard* to raise errors whenever you try to perform operations on vectors with the wrong dimensions. I think I will go with this, instead.
Xenomortis wrote: When choosing between two options, the inconsistent one is rarely better.
Oh, yes; but what I meant was that I created a general purpose shape object that used a point and vector -- stored that point and vector as attributes -- and, when you asked it for the radius of a circle produced from these two values, it would just return the vector. That's not consistency; that's just me being lazy. Radius should be returned as a scalar.

But yeah, I absolutely agree with the notion of consistency over inconsistency.

User avatar
Xanthir
My HERO!!!
Posts: 5186
Joined: Tue Feb 20, 2007 12:49 am UTC
Location: The Googleplex
Contact:

Re: Coding: Fleeting Thoughts

Postby Xanthir » Tue Apr 21, 2015 4:17 pm UTC

Yakk wrote:
Xenomortis wrote:What would you gain by separating out the two concepts (vector vs point)?
For one thing, mathematically there is no difference.

What? No.

Point+Point has no meaning -- points come from an affine set. alpha point + (1-alpha) point is a point.

Vector+Vector is a vector, as is scalar*vector.

Point+vector is a point. Point-point is a vector.

Zero is a distinguished vector, but there are no distinguished points.

Did you mean no arithmatic difference?

Unless there is a distinguished point (the origin) in which case they're identical again.

Even if its not canonically meaningful, most graphics software pretends there's a distinguished point, and so treats points and vectors as identical, with homogenous coordinates distinguishing them.

ETA: Never mind, I confused myself. Homogenous coordinates lets people treat vectors/points and *directions* the same. Points are vectors just by convention, even if it's nonsensical. Points are never *treated as* vectors (I hope...) but they're represented as vectors because there's no arithmetic difference between them as long as you don't perform the meaningless operations.
(defun fibs (n &optional (a 1) (b 1)) (take n (unfold '+ a b)))


Return to “Coding”

Who is online

Users browsing this forum: No registered users and 12 guests