GUI interface writ in Python

A place to discuss the implementation and style of computer programs.

Moderators: phlip, Moderators General, Prelates

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6990
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

GUI interface writ in Python

Postby The Great Hippo » Fri Jul 18, 2014 4:03 pm UTC

Once again, I am working on code for a data-driven roguelike.

I got knocked back in my efforts two years back when I had to move and my grandmother got sick; I started a new, hard job -- bought a house -- and have been engaging in roleplaying games to take the stress off. Things have settled down, though, and I'm getting back into a groove, so I wanted to take another stab at it.

Before I started, I studied a lot of other people's code; specifically, pgu for pygame and OcempGUI. The latter taught me a bit about documentation (almost all the code I'm writing, now, is documented; I've found this habit helps me better organize my own thoughts as I write the code).

Here's my code repository: Adelaide. I'm working extensively on the GUI, right now; rendering isn't a major concern of mine, yet (I just render everything in order every frame). The GUI uses an event system that I've tried to abstract so I can apply it elsewhere (I'm thinking of using the same event system to handle, well, pretty much everything -- game events and rendering, too). The event system supports callbacks connected to specific event types. Widgets sit inside a tree, 'catching' events as they traverse from the root of the tree to its branches and leaves, preventing events from traveling through any nodes that don't have the appropriate filters.

I've kind of hit a wall with my GUI, though; it's a really dumb one: I don't know where to put my mouse checking. At the moment, I've got it loaded in button (under the widgets directory; button inherits from basewidget), but I don't know if it should be loaded in basewidget instead (all widgets, by default, have mouse-checking; other widgets might expand them or use callback functions to associate the mouse-checks with actual replies). I also am feeling really unsure about where to go with my GUI system at this point. I think my code is getting way too big to fit in my head anymore, and that's starting to bug me.

I'm also starting to feel pretty insecure about my code. I've built all of this in about a week, but I really don't know if my solutions are a good long-term approach, and I'd appreciate anyone taking a look at it and helping me sort out where my big mistakes are, structurally. This time my code is pretty thoroughly documented -- I don't think I have many methods or functions that aren't explicitly explained in their docstring. I certainly shouldn't have any undocumented classes.

As before, I know absolutely no one is obligated to hold my hand and look at my code; that being said, any insight from outsiders is greatly appreciated, because I am largely operating in a vacuum, here.

Kian
Posts: 2
Joined: Mon Jul 14, 2014 6:43 pm UTC

Re: GUI interface writ in Python

Postby Kian » Fri Jul 18, 2014 8:44 pm UTC

Well, I'm working on something similar, but in C++. Doing the interface as well right now.

Regarding the issue of the code getting too big: You shouldn't need to understand EVERYTHING your code does at any one time. If the code is getting too big, that's probably a faulty design. Your design should allow you to compartmentalize the different tasks your code is handling and work on one at a time. So you might want to rework your design. I've looked at your repo, and I don't see a design document. Documentation is good, but more important than describing what each class does is describing the overall design of your project. Github has a handy wiki tied to each repo, and I like to use it to design. Here's my attempt at the same thing, for example: github[dot]com/Kynnath/libgui/wiki Got flagged as spam so I had to hide the link.

It's not clean, but it's a work in progress. The important thing is that for all that I've written there, including some sample code, I haven't yet written a single line of code. My advice: try to define interfaces first. How would you actually use the gui code once it is finished? What would the code that calls it look like? That will help you envision the high level design, and some initial data structures. Then start hammering down, with the same method, until you have everything you need plotted out. Coding it at that point is almost trivial. It's a matter of converting from English to python.

I've tried working with just a general idea and diving to code first, and it's generally resulted in the same problem you are describing now. I reach a point where I can't keep going because everything is too tightly coupled, and I'm unsure where to add things. Not knowing where to put your mouse checking seems like the kind of thing a design document would have helped with.

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6990
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: GUI interface writ in Python

Postby The Great Hippo » Sat Jul 19, 2014 3:57 am UTC

Kian wrote:Well, I'm working on something similar, but in C++. Doing the interface as well right now.

Regarding the issue of the code getting too big: You shouldn't need to understand EVERYTHING your code does at any one time. If the code is getting too big, that's probably a faulty design. Your design should allow you to compartmentalize the different tasks your code is handling and work on one at a time. So you might want to rework your design. I've looked at your repo, and I don't see a design document. Documentation is good, but more important than describing what each class does is describing the overall design of your project. Github has a handy wiki tied to each repo, and I like to use it to design. Here's my attempt at the same thing, for example: github[dot]com/Kynnath/libgui/wiki Got flagged as spam so I had to hide the link.

It's not clean, but it's a work in progress. The important thing is that for all that I've written there, including some sample code, I haven't yet written a single line of code. My advice: try to define interfaces first. How would you actually use the gui code once it is finished? What would the code that calls it look like? That will help you envision the high level design, and some initial data structures. Then start hammering down, with the same method, until you have everything you need plotted out. Coding it at that point is almost trivial. It's a matter of converting from English to python.

I've tried working with just a general idea and diving to code first, and it's generally resulted in the same problem you are describing now. I reach a point where I can't keep going because everything is too tightly coupled, and I'm unsure where to add things. Not knowing where to put your mouse checking seems like the kind of thing a design document would have helped with.
Thank you for the response! I actually did write a design document, initially -- but in reading other people's code, I abandoned it for a different design (several times!), until settling upon the current one. I only now realize that once I settled on this design, I did not actually write a design document to describe what I'm trying to do.

I hammered this out in a few hours -- it's essentially the design I've been aiming for. In writing it, I actually found myself solving a few problems I'd been having (such as how to dispose of events I no longer care about), so it's already helped quite a bit! Also, in retrospect, reading a design document (and identifying the internal flaws) is probably a lot easier than reading someone's code. So, maybe it's better for me to post this here and have people tell me if there's anything fundamentally wrong with it:
Spoiler:
DESIGN:

The purpose of this library is to create an extendable, flexible GUI that relies on pygame. Whenever possible, dependencies on pygame should be concentrated in specific modules that can be swapped out with modules that perform the same function with alternative dependencies (such as panda3D).

At the highest level, there are two objects: The EventManager and the EventHandler. But to describe these objects, I must first describe the two lowest level objects that they deal with: The Event and the Listener.

The Event class accepts a signal (an immutable identifier that tells us what type of event it is) and a dictionary (which is assigned to its __dict__). For pygame compatibility, the dictionary can be a pygame.event.Event, in which case the pygame.event.Event's __dict__ will be assigned to the new Event. Events also have a handler attribute, which is set to False; when the handler attribute is set to True, this event is considered 'handled' and can be freely disposed of. An error should be raised if the dictionary you pass to an event includes a 'handled' or 'signal' key (as this would interfere with these attributes).

The Listener class has two attributes; signals and callbacks. Signals is a set of event signals that this object will accept; callbacks is a dictionary of signal : List-of-Functions pairs which are applied when the appropriate signal is received by the Listener (firing all the functions in the corresponding list). Listeners may also emit events (ie, create them).

Back to the two highest level objects: The EventManager class has a queue of events and a list of EventHandlers currently registered with it. When the EventManager accepts a list of events, it processes them and cycles them through each EventHandler that has registered with it (stopping if the event's handled attribute is set to True, and only dropping an event in an EventHandler if the EventHandler has the event's signal in its signals attribute).

The EventHandler has a set of signals it will receive. It registers with the EventManager (to receive events), accepts events (and drop them into whatever Listeners the EventHandler administrates over), and manages its particular systems. Ideally I'd like EventHandlers to be threadsafe, but first I need to understand what that actually MEANS ( >_> ). Essentially, all major 'systems' that deal with events inherit from the EventHandler -- the GuiHandler being one of those systems.

That's the high end stuff. Now, to move down to the next level, I need to talk about graphs and wrappers.

There are currently two graph objects; TreeGraph and TreeNode. TreeGraph is a graph for TreeNodes; TreeNodes are the occupants of the tree (with the TreeGraph itself acting as the solitary 'root'). TreeNodes can only have one parent, but can have an arbitrary number of (ordered) children. Whenever you create a TreeNode, you name the parent it belongs to, which prompts the TreeNode to add itself to its parent's list of children.

Wrappers are essentially classes that handle the pygame end of things; the important ones are RectWrapper (which has an internal pygame.Rect object and uses virtual attributes to make it easy to access) and Drawable (which includes a surface attribute, an image attribute, and convenience methods for drawing to surfaces).

Now, take an object and make it a subclass of TreeNode (meaning it's a component of a TreeGraph), RectWrapper (meaning it has an internal rect and has handy virtual attributes to modify that rect), and Drawable (meaning it has a surface, an image, and convenience functions for drawing). Add in Listener (meaning it can receive events, emit events, store callbacks, and respond to events with callbacks), and you now have BaseWidget: The basic GUI object.

The BaseWidget adds a little extra functionality, too: Whenever you add signals to the set of signals a BaseWidget accepts, it adds those signals to the set of signals all of its parents accept (thereby populating up the list). This includes the WidgetTree (which inherits from TreeGraph; it's just a TreeGraph with some extra functionality to help it interact with BaseWidget) itself, which should, upon adding a new signal to its own acceptable set of signals, ALSO add that signal to the set of signals that its EventHandler accepts. BaseWidgets also need to perform translations -- whenever you move its internal rect (inherited from RectWrapper), it updates its children's position appropriately.

Finally, I can talk about GuiHandler, which inherits from EventHandler. GuiHandler is essentially a state engine that swaps out WidgetTrees depending on its current state. When swapped, a pointer to the old WidgetTree is stored in the new WidgetTree, so we can return to the previous WidgetTree (start menu -> game -> save menu, then back to game, then back to start menu, then quit interface). When the GuiHandler receives an event from the EventHandler, it drops the event into
the currently active WidgetTree.

The WidgetTree then 'passes' this event (via the pass_event method) to its children (in order -- remember, children of TreeGraphs and TreeNodes are ordered lists, not sets!), who pass it to their children, etc -- relying on a DEPTH FIRST SEARCH. When a child is found who doesn't have the event's signal in its set of acceptable signals, the event does not move beyond that child. When an event's handled attribute is set to True, the event immediately stops being passed. Each child who successfully 'receives' an event has its notify method fired, with the event as an argument. The notify method activates any signal callback functions connected to that signal, firing them as appropriate.

Extending the library would involve having objects inherit from BaseWidget with additional functionality. Hopefully, this system is generalized enough that I can create other subclasses of the EventHandler that benefit from the Event system without much difficulty (having the 'Listeners' they monitor over simply inherit from the Listener class and any other classes as appropriate).


Writing this out also caused me to notice a few problems I hadn't noticed, and realize there are a couple of things I've put in the code that I don't actually need. But I'm still a little hesitant about putting the mouse catching stuff in BaseWidget -- though the more I think about this, the more I think it's the right move.

Any thoughts on the design document above -- whether or not it's a structurally sound design, where some points of weakness might be, or even if it's completely unintelligible -- would be appreciated!

EDIT: Ah, I just realized -- BaseWidget should just fire off its signal callback functions on notify -- because I might want widgets that do that (do nothing but act as containers for signal callbacks). Like an invisible widget that catches a particular key press, then fires a function in response. So, rather, BaseWidget doesn't do anything to catch mouse input; I'll create a new class that inherits from BaseWidget and, on its notify method, deals with mouse input and generates appropriate events for it -- and any widget that has to deal with the mouse in any way will inherit from that widget.

User avatar
Jplus
Posts: 1709
Joined: Wed Apr 21, 2010 12:29 pm UTC
Location: Netherlands

Re: GUI interface writ in Python

Postby Jplus » Sat Jul 19, 2014 8:41 pm UTC

You explain your design very clearly, that is important.

While you explain very well what your framework is supposed to do and how it does it, I'm missing a "why". In particular, the design seems rather complicated as it is and it makes me wonder why it couldn't be done with fewer components.

Perhaps you are treating two orthogonal problems as if they are one, i.e. making an abstraction of rendering engines on the one hand and taking care of event handling on the other hand. In that case, it might be easier to oversee for yourself and for others if you split the effort in two smaller frameworks. Possibly with a third, high-level game framework that builds on the other two.

Perhaps those responsibilities are already more or less isolated within your framework. In that case, maybe all you need to do is to make the separation more explicit.
"There are only two hard problems in computer science: cache coherence, naming things, and off-by-one errors." (Phil Karlton and Leon Bambrick)

coding and xkcd combined

(Julian/Julian's)

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6990
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: GUI interface writ in Python

Postby The Great Hippo » Sat Jul 19, 2014 10:10 pm UTC

Jplus wrote:Perhaps you are treating two orthogonal problems as if they are one, i.e. making an abstraction of rendering engines on the one hand and taking care of event handling on the other hand. In that case, it might be easier to oversee for yourself and for others if you split the effort in two smaller frameworks. Possibly with a third, high-level game framework that builds on the other two.

Perhaps those responsibilities are already more or less isolated within your framework. In that case, maybe all you need to do is to make the separation more explicit.
Hm. I'm trying to abstract my solution for creating a GUI interface with tree graphs (by dropping events into those tree graphs and allowing GUI components to 'catch' them) to the point where it can also serve as a solution for other problems, such as rendering, game logic, and sound. Essentially, I'm trying to create a global event system.

I've been warned off of global systems in the past, though, and probably with good reason. If I paired down my event system to deal with only the GUI, it would probably simplify a lot of this.

EDIT: Also, I just realized tonight that the EventHandler object is really just a Listener object with some extra functionality; it's a Listener that distributes events to other Listeners (probably as a result of its notify method). So I don't actually need an EventHandler; I can just have Listeners register with the EventManager and have subclasses of Listeners distribute events they receive to other Listeners (such as the widgets in a tree graph).

EDIT-EDIT: Also, I wrote out a design doc using github's handy wiki (thanks Kian!) available here; at the moment, I'm still thinking of going forward with a generalized event system.

Breakfast
Posts: 117
Joined: Tue Jun 16, 2009 7:34 pm UTC
Location: Coming to a table near you

Re: GUI interface writ in Python

Postby Breakfast » Mon Jul 21, 2014 6:13 pm UTC

*Warning* I haven't looked at your code.

EDIT: Also, I just realized tonight that the EventHandler object is really just a Listener object with some extra functionality; it's a Listener that distributes events to other Listeners (probably as a result of its notify method). So I don't actually need an EventHandler; I can just have Listeners register with the EventManager and have subclasses of Listeners distribute events they receive to other Listeners (such as the widgets in a tree graph).


What you're getting at here is entirely true. Eventing systems are a specific application of the Observer pattern. JPlus is making a good suggestion when he says that you might want to separate your abstractions. My two cents is that you might want to have *Observers (such as InterfaceObserver, GameLogicObserver, ...) if their responsibilities are fairly different.

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6990
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: GUI interface writ in Python

Postby The Great Hippo » Thu Jul 31, 2014 12:58 pm UTC

I appreciate the help! I've been working on a major offline rewrite for the past few days, which (when done) I'll be uploading into the associated github.

I've spent some time reading up on messaging and event-driven architecture and I think I have a better, simpler solution that focuses on the GUI interface rather than a broad, global solution. I've written a design document that I've uploaded to the site's wiki already, but in instituting it, I've already noticed some problems and have made several major changes (once I'm done the code, I'll re-write the design document to reflect the new design).

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6990
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: GUI interface writ in Python

Postby The Great Hippo » Tue Aug 05, 2014 6:22 pm UTC

I've been working on this pretty diligently offline; I haven't uploaded to github yet because the amount of changes I've made are so vast that I think it's going to end up just getting completely overwritten. I've been reading a lot about dispatchers, publish-subscribe systems, observer-subject patterns, and messaging patterns in general; I feel like I've learned quite a bit.

Below, I'm posting my solution for the GUI system I want to mess with; CallbackCapable and CallbackSignature objects. Eventually, they'll be controlled by a Dispatcher object which will be responsible for sending messages to them.
Spoiler:

Code: Select all

"""Contains the CallbackCapable and CallbackSignature classes; the former
enables its subclasses to trigger callbacks upon receiving appropriate signals
from appropriate senders, while the latter provides a simple object to
safely remove callbacks placed on CallbackCapable objects.

NOTE: 'Any' is an (unenforced) singleton object imported from dispatcher.locals
that's used for cases where a callback should be triggered by any sender or
signal.
"""

from dispatcher.locals import *

class CallbackSignature:
    def __init__(self, signal, sender, func, params):
        """CallbackSignature(signal, sender, func) -> CallbackSignature

        Instantiate a CallbackSignature object, which provides a simple
        signature to use for removing callbacks assigned to CallbackCapable
        objects. When a CallbackCapable object has a callback associated with
        it, it returns a CallbackSignature object.
        """
        self.signal = signal
        self.sender = sender
        self.func = func
        self.params = params
    def __repr__(self):
        return "%s(%r, %r, %r)" % (self.__class__.__name__, self.signal,
                                self.sender, self.func)

class CallbackCapable:

    def __init__(self):
        """CallbackCapable() -> CallbackCapable

        Instantiate a CallbackCapable object. These are objects which can
        respond to notices (messages) sent to their run_callbacks method with
        appropriate callbacks (when these messages have the right signature
        and sender).

        Attributes:
            _callbacks - A dictionary with signals as its keys and another
            dictionary as its values. The second dictionary contains senders
            as its keys and a list of tuples (a callback and its parameters)
            as its values.
            dispatchers - The set of dispatchers currently assigned to send
            messages to this object's notify method.
        """
        self._callbacks = {Any : {Any : []}}
        self.dispatchers = set()
    def __repr__(self):
        return '%s(%r, %r)' % (self.__class__.__name__, self._callbacks,
                            self.dispatchers)
    def connect_callback(self, signal, sender, callback, *params):
        """CallbackCapable(signal, sender, callback, *params) ->
        CallbackSignature

        Connect a callback function to a signal and a sender. If signal or
        sender are set to 'None', their values become 'Any', and the callback
        will be triggered on any signal or sender.
        """
        if not is_hashable(signal): raise TypeError('Signal not hashable.')
        if not is_hashable(sender): raise TypeError('Sender not hashable.')
        if not callable(callback): raise TypeError('Callback not callable.')
        if signal is None: signal = Any
        if sender is None: sender = Any
        if signal in self._callbacks:
            if sender in self._callbacks[signal]:
                self._callbacks[signal][sender].append((callback, params))
            else:
                self._callbacks[signal][sender] = [(callback, params)]
        else:
            self._callbacks[signal] = {Any : [], sender : [(callback, params)]}
#        self._update_dispatchers(signal)
        return CallbackSignature(signal, sender, callback, params)
    def remove_callback(self, cb_sig):
        """CallbackCapable.remove_callback(cb_sig) -> None

        Removes a callback, relying on a CallbackSignature object to do it.
        Also cleans up after itself, culling any entries in _callback that
        are empty (but aren't associated with Any).
        """
        signal, sender, callback = cb_sig.signal, cb_sig.sender, cb_sig.func
        self._callbacks[signal][sender].remove((callback, cb_sig.params))
        if self._callbacks[signal][sender] == []:
            if sender is Any: return
            else: self._callbacks[signal].pop(sender)
            if self._callbacks[signal] == {Any: []}:
                if signal is Any: return
                self._callbacks.pop(signal)
    def run_callbacks(self, message):
        """CallbackCapable.run_callbacks(message) -> None

        Fire all callbacks associated with a specific signal and sender.
        """
        signal, sender = message.signal, message.sender
        if signal in self._callbacks:
            if sender in self._callbacks[signal]:
                for cb_tuple in self._callbacks[signal][sender]:
                    self.run_callback(cb_tuple)
            for cb_tuple in self._callbacks[signal][Any]:
                self.run_callback(cb_tuple)
        if sender in self._callbacks[Any]:
            for cb_tuple in self._callbacks[Any][sender]:
                self.run_callback(cb_tuple)
        for cb_tuple in self._callbacks[Any][Any]:
            self.run_callback(cb_tuple)
    def run_callback(self, cb_tuple):
        """CallbackCapable.run_callback(cb_tuple) -> None

        Run a specific callback tuple (the first value is the callback, the
        second, its parameters).
        """
        callback = cb_tuple[0]
        params = cb_tuple[1]
        callback(*params)

Usage example:

Code: Select all

>>> def foo():
>>>     print ('hello')
>>>
>>> def foo2(x):
>>>     print (x)
>>>
>>> def foo3(x, y):
>>>     print (x)
>>>     print (y)
>>>
>>> class Message:
>>>     def __init__(self):
>>>         self.signal = 'mouse_rclick'
>>>         self.sender = 'button'
>>>
>>> obj = CallbackCapable()
>>> cb1 = obj.connect_callback('mouse_rclick', 'button', foo)
>>> cb2 = obj.connect_callback('mouse_rclick', 'button', foo2, 'hi')
>>> cb3 = obj.connect_callback('mouse_rclick', 'button', foo3, 'bluh', 'blah')
>>> m = Message()
>>> obj.run_callbacks(m)
hello
hi
bluh
blah
>>>
What I'm aiming for is a system where I can use callbacks to set up additional callbacks -- so I can have a callback that creates another callback, or -- more importantly -- a callback that *removes* another callback. That way, I can avoid having to juggle states in my GUI objects (controlling a GUI object's behavior purely through its callbacks).

So, ideally, for something like dragging a window object when you click on its handle bar, you'd just need...

Code: Select all

handle_bar = CallbackCapable()
window = CallbackCapable()
cb1 = window.connect_callback('mouse_lclick', handle_bar, window.connect_callback, 'mouse_move', Any, window.move_to_mouse)
cb2 = window.connect_callback('mouse_lrelease', Any, window.remove_callback, cb1)
...and that should cause the window to respond to a click on the handle bar by immediately linking the window's position with the mouse's motion (whenever you move the mouse, that fires a callback on the window to move it to the mouse's position), and also destroying that callback the moment you lift the mouse button up (release the drag).

I'm still a little leery of having callbacks in my callbacks, though, because I don't yet understand how nested parameters would work in my connect_callback function (I need to test and find out!). Hence why my theoretical 'move_to_mouse' method accepts no parameters -- either way, I'm going to test this a bit more thoroughly to make sure everything's working as it should, and then I'm going to move back to setting up dispatchers.

Any thoughts are appreciated!

EDIT: I just tested this, and realized why it doesn't work; the nested callback isn't cb1, so when I go to remove it, I'm removing the callback that put it there -- rather than the nested callback. I'll need to think about this for a bit.

EDIT-EDIT: Oh, easy solution; check and see if the callback is actually a 'connect_callback' method, and if so, return a tuple -- the original callback's signature and the signature of the nested callback. Also, include error code to catch if someone's trying to nest a callback inside the nested callback -- as that's just kind of ridiculous.

Breakfast
Posts: 117
Joined: Tue Jun 16, 2009 7:34 pm UTC
Location: Coming to a table near you

Re: GUI interface writ in Python

Postby Breakfast » Thu Aug 07, 2014 5:17 pm UTC

What you're doing seems decent but I'm not really good at reading python so I'm probably not the best judge. What I want to try to accomplish in this post is to describe the problem in a code agnostic way to see if we're on the same page. If I'm rambling or unclear please just let me know!

Let's start off with thinking about a button. Every so often the button is going to shout out, "I've just been clicked!" The important concept is that this happens again and again through time. Using some terminology, the button's click is Observable; it's a Publisher.

The second piece of this pattern is you (possibly your CallbackCapable). The button is always shouting out whenever it's clicked, but so far no one is paying attention. This is where you come in. You are the the Observer, the Subscriber, the Listener. The button's click is something your interested in watch; so you sign up to watch it. The really interesting part is the callback. So now you're listening to the button shout out every time it's click but that's fairly boring. The callback is a piece of code that you specify to run every time that button gets click.

I'm think you've already understood what I've just explained and it's just my lack of understanding of python that's throwing me off.

To bring in some of your code now:

Code: Select all

>>> cb1 = obj.connect_callback('mouse_rclick', 'button', foo)

This looks like what I'm explaining as obj is the Observer, 'mouse_rclick' is the Observable, and foo is the callback.

Code: Select all

>>> obj.run_callbacks(m)

This is throwing me off. In this case is the Observer is running all of the callback or just those callbacks for the right mouse click? Technically Observers can watch for different events. Is Message supposed to be simulating the mouse notifying Observers that the right button has been clicked? I'm guessing that this is the case but I just wanted to clarify.

*Immediate Edit*
I'd be shocked if python didn't have some kind of built in handling for events. It's awesome that you're doing a deep dive into the topic and getting experience with the underlying concepts and fundamentals but after you do it's probably a better idea to use the built in facilities. They'll handle all kinds of edge cases you or I might miss.

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6990
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: GUI interface writ in Python

Postby The Great Hippo » Thu Aug 07, 2014 7:19 pm UTC

I beg your pardon! I have a bad tendency to talk about an idea without presenting any of the key components required for it to function:

Pygame's event system produces events (like a mouse move, mouse click, or key press) which are dropped into my GUI widget tree. As these events traverse the tree, each widget examines them and determines if they're relevant to the widget and whether or not they should be marked as 'handled' (IE, done with; prevented from continuing the traversal). So, for example, a mouse click goes down the widget tree until it hits something (like a button) that marks it as 'handled'. That button then requests a message be made and sent -- a signal (string), sender (the button itself), and dictionary (any special information we'd like the message to contain). The signal would probably be something like 'left_click_widget', and the sender would be the widget itself. The dictionary might contain some information relevant to the mouse's position.

(As a side note, the way I have this structured, a widget does not need to completely consume a message to emit a message; you can have a widget that responds to a click with a message, and allows the click to continue traversing the tree)

This requested message is created and directed to all other widgets (and any relevant parties) via a Dispatcher (an object I'm working on creating now!), which manages -- and distributes -- messages. Dispatchers are the publishers; they have subscribers (who register with the Dispatcher) who request a set of signals they're interested in (along with the methods -- or functions! -- they want messages of these signals to be routed to). Most of the subscribers will be CallbackCapable objects, but the way I'm trying to structure it, this is not necessary -- Dispatchers can be set up to send messages to any callable object (I want both Dispatchers and CallbackCapables to be completely independent of one another; they are both dependent on Message, but Message is a very simple object).

All the widgets are CallbackCapables, so -- when I have that button that requests a message with a 'left_click_widget' signal (along with the widget that was clicked), the Dispatcher creates and sends this message to every object registered to receive 'left_click_widget' signals (which will end up being every widget, by default), thereby notifying them that this widget has been left-clicked. That triggers any and all of their callbacks associated with left_click_widget signals from that sender (optionally, callback triggers can be set to 'None' to fire in cases of ANY signal or ANY sender).

I think this would make CallbackCapables my Observers; but it also makes them the subjects I'm observing? I'm not sure. I've realized in writing this that I have completely lost track of who is observing what (in the sense that that I don't know where my observers end and my subjects begin). Maybe that's a bad thing.

Anyway, here's my updated CallbackCapable code; I've gotten it to support callback connections that add in (and discard!) other callback connections:
Spoiler:

Code: Select all

"""Contains the CallbackCapable and CallbackSignature classes; the former
enables its subclasses to trigger callbacks upon receiving appropriate signals
from appropriate senders, while the latter provides a simple object to
safely remove callbacks placed on CallbackCapable objects.
"""

from dispatcher.locals import *
from dispatcher.errors import *

class CallbackSignature:
    def __init__(self, signal, sender, func, params):
        """CallbackSignature(signal, sender, func, params) -> CallbackSignature

        Instantiate a CallbackSignature object, which provides a simple
        signature to use for removing callbacks assigned to CallbackCapable
        objects. When a CallbackCapable object has a callback associated with
        it, it returns a CallbackSignature object.

        Parameters:
            signal - The signal that triggers the callback.
            sender - The sender that triggers the callback.
            func - The callable of the callback.
            params - The parameters to pass to the callable.

        Attributes:
            cb_tuple - A tuple containing the callable as its first value, and
            the parameters (enclosed in a tuple) as its second value.
        """
        self.signal = signal
        self.sender = sender
        self.func = func
        self.params = params
        self.cb_tuple = (func, params)
    def __repr__(self):
        return "%s(%r, %r, %r, %r)" % (self.__class__.__name__, self.signal,
                                self.sender, self.func, self.params)

class CallbackCapable:
    def __init__(self):
        """CallbackCapable() -> CallbackCapable

        Instantiate a CallbackCapable object. These are objects which can
        respond to notices (messages) sent to their run_callbacks method with
        appropriate callbacks (when these messages have the right signature
        and sender).

        Attributes:
            _callbacks - A dictionary with signals as its keys and other
            dictionaries as its values. The internal dictionary contains
            senders as its keys and a list of tuples (callbacks and their
            parameters) as its values.
        """
        self._callbacks = {}
    def __repr__(self):
        return '%s(callbacks: %r)' % (self.__class__.__name__,
                                                        self._callbacks)
    def __str__(self):
        return '%s()' % (self.__class__.__name__)
    def connect_callback(self, signal, sender, callback, *params):
        """CallbackCapable(signal, sender, callback, *params) ->
        CallbackSignature or tuple

        Connect a callback function to a signal and sender, returning a
        CallbackSignature object (to allow for easy removal of the callback).
        If the callback is another connect_callback method, return a tuple
        containing both CallbackSignatures. Raise an error if the second
        connect_callback method calls *another* connect_callback method.

        NOTE: Sending 'None' for either signal or sender will result in the
        run_callbacks method assuming these values are correct for the purposes
        of triggering a callback.
        """
        self.__error_checking(signal, sender, callback, *params)
        if signal in self._callbacks:
            if sender in self._callbacks[signal]:
                if (callback, params) not in self._callbacks[signal][sender]:
                    self._callbacks[signal][sender].append((callback, params))
            else:
                self._callbacks[signal][sender] = [(callback, params)]
        else:
            self._callbacks[signal] = {sender : [(callback, params)]}
        if callback.__name__ is 'connect_callback':
            try: tail_end_params = params[3]
            except IndexError: tail_end_params = ()
            return (CallbackSignature(signal, sender, callback, params),
                    CallbackSignature(params[0], params[1], params[2],
                    tail_end_params))
        return CallbackSignature(signal, sender, callback, params)
    def disconnect_callback(self, cb_sig):
        """CallbackCapable.disconnect_callback(CallbackSignature) -> None

        Disconnect a callback, relying on a CallbackSignature object to do it.
        Also cleans up after itself, culling any entries in _callback that
        are empty.
        """
        signal, sender, cb_tuple = cb_sig.signal, cb_sig.sender, cb_sig.cb_tuple
        self._callbacks[signal][sender].remove(cb_tuple)
        if self._callbacks[signal][sender] == []:
            self._callbacks[signal].pop(sender)
            if self._callbacks[signal] == {}:
                self._callbacks.pop(signal)
    def run_callbacks(self, message):
        """CallbackCapable.run_callbacks(message) -> None

        Fire all callbacks associated with a message. If a signal or sender is
        set to None, presume association.
        """
        signal, sender = message.signal, message.sender
        cb_tuples = []
        if signal in self._callbacks:
            if sender in self._callbacks[signal]:
                cb_tuples += self._callbacks[signal][sender]
            if None in self._callbacks[signal]:
                cb_tuples += self._callbacks[signal][None]
        if None in self._callbacks and sender in self._callbacks[None]:
            cb_tuples += self._callbacks[None][sender]
        if None in self._callbacks and None in self._callbacks[None]:
            cb_tuples += self._callbacks[None][None]
        cb_tuples = cull_sequence(cb_tuples)
        for cb_tuple in cb_tuples:
            self.__run_callback(cb_tuple)
    def __run_callback(self, cb_tuple):
        """CallbackCapable.run_callback(cb_tuple) -> None

        Run a specific callback tuple (the first value is the callback, the
        second, its parameters).
        """
        callback = cb_tuple[0]
        params = cb_tuple[1]
        callback(*params)
    def __error_checking(self, signal, sender, callback, *params):
        """Basic error-catching. For internal use only."""
        if not is_hashable(signal): raise CallbackError('Signal not hashable.')
        if not is_hashable(sender): raise CallbackError('Sender not hashable.')
        if not callable(callback): raise CallbackError('Callbacks must be \
                                                        callable objects.')
        if callback.__name__ is 'connect_callback':
            if not callable(params[2]):
                raise CallbackError('Callbacks must be callable objects.')
            if params[2].__name__ is 'connect_callback':
                raise CallbackError('You cannot double-nest callbacks.')


class Message:
    def __init__(self, signal, sender, attr = None):
        """Message(sender, signal, attr = None) -> Message

        Instantiate a Message object. Message objects are how information is
        sent in Adelaide. Raises an exception if you try to assign a
        non-hashable value to the signal or sender parameters.

        Parameters:
            sender - The sender of the message.
            signal - The signal (type) for the message. Hashable objects only.
            attr - Either a dictionary or an object with an internal __dict__.
        """
        if attr is None: attr = {}
        if hasattr(attr, '__dict__'): self.__dict__ = attr.__dict__
        else: self.__dict__ = attr
        if not is_hashable(signal): raise MessageError('Signal not hashable.')
        if not is_hashable(sender): raise MessageError('Sender not hashable.')
        if signal is None or sender is None:
            raise MessageError('Signals and senders cannot be None.')
        self.signal = signal
        self.sender = sender
        self.handled = False
    def __repr__(self):
        attr = self.__dict__.copy()
        attr.pop('signal')
        attr.pop('sender')
        return '%s(%r, %r, %r)' % (self.__class__.__name__, self.signal,
                                    self.sender, attr)

def foo():
    print ('foo')

def foo1(x):
    print (x)

window = CallbackCapable()
handle_bar = CallbackCapable()
m_click = Message('click', handle_bar)
m_move = Message('move', handle_bar)
m_release = Message('release', handle_bar)

tick_callback = window.connect_callback(None, handle_bar, foo1, 'tick')
callbacks = window.connect_callback('click', handle_bar,
                                    window.connect_callback, 'move', None, foo)
callback = window.connect_callback('release', handle_bar,
                                    window.disconnect_callback, callbacks[1])
And here's a quick demonstration of what it allows:
Spoiler:

Code: Select all

*** Remote Interpreter Reinitialized  ***
>>>
>>> window.run_callbacks(m_move)
tick
>>> window.run_callbacks(m_click)
tick
>>> window.run_callbacks(m_move)
foo
tick
>>> window.run_callbacks(m_click)
tick
>>> window.run_callbacks(m_move)
foo
tick
>>> window.run_callbacks(m_release)
tick
>>> window.run_callbacks(m_move)
tick
>>> window.run_callbacks(m_click)
tick
>>> window.run_callbacks(m_move)
foo
tick
>>>
So what's happening in the above code is...

I'm creating some messages, CallbackCapable objects, and callbacks (linked to those CallbackCapables). I'm demonstrating that I can create a callback on a CallbackCapable object that connects another callback (so I can have a callback that says, 'when you click this widget, create another callback that makes it so when you move the mouse, run function foo), then create another callback on a CallbackCapable object that disconnects that callback (so I can have a callback that says, 'when you release the mouse button, disconnect that callback that makes it so when you move the mouse, run function foo').

The goal is to work around having to juggle states and what not on my GUI objects (I imagine the alternative solution here would be to deal with things like 'STATE_DRAGGABLE'); to drastically alter the behavior of my widgets using callbacks to add and remove those behaviors dynamically rather than having to hard-code the behavior into the widget itself. So rather than having a draggable widget, or a bunch of special code for handling widget cases, I can handle pretty much everything re: dragging an object with three lines of code and a 'move_widget' function that I was probably going to want to have anyway.

One pseudo-problem (as in I don't know if it's actually a problem) is that callbacks currently do not get access to the messages that triggered them -- that might be relevant later on (in fact, it was the whole reason for messages having a dictionary! So callbacks could receive special data beyond just the signal and sender!).

To solve this pseudo-problem, I might want to have some code in my '__run_callback' method that checks to see if the callback function has a 'message = None' parameter, and if it does, send the message, too. That way, callback functions that need the message to do their job always get it, and ones that don't care never do.
Breakfast wrote:I'd be shocked if python didn't have some kind of built in handling for events. It's awesome that you're doing a deep dive into the topic and getting experience with the underlying concepts and fundamentals but after you do it's probably a better idea to use the built in facilities. They'll handle all kinds of edge cases you or I might miss.
I don't think Python has any built in event handling. I've been reading through a lot of optional Python libraries that deal with events and messages, though -- pydispatcher, for example, is where I got the idea for having a dispatcher that can send messages to any callable, rather than just a default method (which means that objects who register with the dispatcher don't need to inherit from anything, and don't need any special methods to deal with the dispatcher).

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6990
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: GUI interface writ in Python

Postby The Great Hippo » Fri Aug 08, 2014 1:54 am UTC

Also, I just had a small but critical epiphany while reading another Python tutorial: My callback system is essentially operating as a clumsy implementation of Python's lambda system.

IE, I just realized that Python lambdas (which I have failed to completely understand up until this point) can be used to wrap function x + parameters into an anonymous function that you can fire later -- which is pretty much precisely what I am doing (my cb_tuple is a tuple with a function and an interior tuple of parameters to pass to that function).

I need to read a bit more about lambdas and figure out if this has any implication for my code; I think my code operates just fine without lambdas, but there may be good reason to put them in (for example, it might make nested callbacks -- callbacks connecting callbacks connecting callbacks -- much easier, which could lead me to take out the code that throws an exception when you try to nest callbacks more than once).

Breakfast
Posts: 117
Joined: Tue Jun 16, 2009 7:34 pm UTC
Location: Coming to a table near you

Re: GUI interface writ in Python

Postby Breakfast » Fri Aug 08, 2014 11:49 am UTC

The Great Hippo wrote:(I want both Dispatchers and CallbackCapables to be completely independent of one another; they are both dependent on Message, but Message is a very simple object).

This is an awesome thing to try and accomplish. You definitely want them to be independent of each other so that you'll have the ability to swap one out without the other breaking. Once again, I don't know python but does it have a concept like interfaces? If so, you would be able to have Message implement an interface and then the Dispatcher and CallbackCapables would accept the interface. That way you're not tightly coupled to Message's implementation either.
The Great Hippo wrote:I think this would make CallbackCapables my Observers; but it also makes them the subjects I'm observing? I'm not sure. I've realized in writing this that I have completely lost track of who is observing what (in the sense that that I don't know where my observers end and my subjects begin). Maybe that's a bad thing.

CallbackCapable is the observer. It sounds like Dispatcher is the obervable because it collects all of the "real" observables and notifies the system when something happens. If the distinction is unclear in code, I suggest formalizing it a bit more. If the distinction is unclear in your mind, I hope this gives you some confidence that you're on the right path!
The Great Hippo wrote:The goal is to work around having to juggle states and what not on my GUI objects (I imagine the alternative solution here would be to deal with things like 'STATE_DRAGGABLE'); to drastically alter the behavior of my widgets using callbacks to add and remove those behaviors dynamically rather than having to hard-code the behavior into the widget itself. So rather than having a draggable widget, or a bunch of special code for handling widget cases, I can handle pretty much everything re: dragging an object with three lines of code and a 'move_widget' function that I was probably going to want to have anyway.

I'd hate to throw more complexity on when you're already tackling a difficult concept so feel free to skip this paragraph if you want.
Spoiler:
Your goal makes me think of the Presenter in the Model-View-Presenter (MVP) pattern. The presenters role is to mediate between the view and model. As the name suggests: the presenter takes data from a model, examines it and makes decisions, potentially transforms it in some way, and shoves that transformation off to the view. The view might then fire off some events which the present will respond to, make decisions, transform data, and then shove it all back to the model. In this way MVP is a composite of several design patterns. Read more here!

You've got the right ideas and seem to be making a lot of progress! I haven't thought to a great deal on the lambda thing yet but it seems plausible.

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6990
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: GUI interface writ in Python

Postby The Great Hippo » Fri Aug 08, 2014 11:00 pm UTC

I really appreciate the support and feedback, btw; I am constantly struggling to build an accurate image of myself re: where I am and what I'm doing code-wise, and talking it out with other people helps me do that.
Breakfast wrote:This is an awesome thing to try and accomplish. You definitely want them to be independent of each other so that you'll have the ability to swap one out without the other breaking. Once again, I don't know python but does it have a concept like interfaces? If so, you would be able to have Message implement an interface and then the Dispatcher and CallbackCapables would accept the interface. That way you're not tightly coupled to Message's implementation either.
Luckily, Python is a duck-type language, which means as long as an object quacks like a duck you're free to treat it like a duck. Which means as long as an object has a 'signal' and 'sender' attribute, it can trigger a callback from CallbackCapable (as these are the only things CallbackCapable objects look for!). When I said CallbackCapable is dependent on Message, I misspoke; it's really just dependent on any object with those two attributes.

That being said, your comment here gave me a potential solution for a problem I have with Messages (see way below!), and thinking/learning about interfaces got me to thinking about how to provide easy functionality for the Dispatcher (also see below!).
Breakfast wrote:CallbackCapable is the observer. It sounds like Dispatcher is the obervable because it collects all of the "real" observables and notifies the system when something happens. If the distinction is unclear in code, I suggest formalizing it a bit more. If the distinction is unclear in your mind, I hope this gives you some confidence that you're on the right path!
That makes a lot more sense, I think -- Dispatchers are where the observable data gathers; it delivers that observable data to the observers -- those objects registered with the Dispatcher. I can make this more explicit by describing the objects that register with a Dispatcher as 'observers', and objects which send message requests to the Dispatcher as 'observables', or the specific push itself as an 'observation'.

Part of the confusion for me was the realization that an observable can also be an observer (in fact, I've had this problem come up in previous code -- where an object pushed a message request, then received the generated message from the dispatcher! -- which struck me as a waste), but what helps is realizing that an observable is not necessarily an observer.

Thinking about this, and the interface you mentioned above for messages, has gotten me to thinking that producing an interface for objects to use Dispatchers -- an interface they get from a Dispatcher itself, when that object registers with the Dispatcher (registering returns an interface object 'attuned' to the specific Dispatcher) -- could be a good route to go. Observers register with a Dispatcher and add that Dispatcher's interface as a component to themselves (observer.my_dispatcher = dispatcher.register(observer.run_callbacks), signals).

(As another aside, it occurs that the methods the dispatcher sends messages to are the ACTUAL observers -- not the objects these methods are bound to! Which actually makes the interface object even more reasonable, as a means for an object that 'owns' an observer to interact both with that observer and the dispatcher that's sending observations to it)
Breakfast wrote:Your goal makes me think of the Presenter in the Model-View-Presenter (MVP) pattern. The presenters role is to mediate between the view and model. As the name suggests: the presenter takes data from a model, examines it and makes decisions, potentially transforms it in some way, and shoves that transformation off to the view. The view might then fire off some events which the present will respond to, make decisions, transform data, and then shove it all back to the model. In this way MVP is a composite of several design patterns. Read more here!
The MVP pattern was actually one of the ones I've been trying to simulate (I just learned about it -- well, just finally started understanding it -- a month or two ago), so hearing that my solution sounds like it's subscribing to that pattern is actually quite a relief -- as that is part of my goal!
Breakfast wrote:You've got the right ideas and seem to be making a lot of progress! I haven't thought to a great deal on the lambda thing yet but it seems plausible.
Thank you! After reading a bit more about the lambda implementation in Python, though, I've started to suspect it might be limited -- as ad hoc and clumsy as my solution is, I don't think switching over to lambda would net me any real gains. In fact, I'm going to test later today to see if creating callbacks that connect callbacks that connect callbacks that connect callbacks (etc) actually creates any problems (I suspect it does not; I mostly made it an error just because I wasn't sure if it would even work).

---

As another aside, I'm encountering a problem with messages that might be resolved with either decorators or interfaces. Messages can be assigned extra data besides the signal and sender; they accept a dictionary, which they then assign to their attributes (so a dictionary with 'mouse_position' and an (x, y) tuple would become 'message.mouse_position = (x, y)'). This can be useful for callbacks that want a little extra data to work with beyond the signal and sender; for example, if I want to use a callback to move a window, I might want to have the message that triggers it include values describing where it should move (in the case of dragging, I want it to move to where the mouse has moved!). So, a signal like 'mouse_move' should have data regarding where the mouse has moved to (and where its previous position was).

There are two problems with this, though: First, that means I need to pass the message to my callbacks, which means callbacks have to be set to accept messages as values. And second, that also means I need to have a way of guaranteeing what types of 'meta-data' a message of a given signal will contain (otherwise, my callbacks might look for data that isn't even there -- like looking for mouse position information in a 'key_press' signal message).

I've got some ideas about how to solve this problem with decorators, but they all kind of suck because they create a dependency -- any callable you plan to use as a callback must be decorated, and probably always by the same function. But I think otherwise, decorations are a very simple solution; if I don't mind decorating each and every function I might use as a callback with the @callback decorator, it should be trivial to figure out a way to pass extra message data to any callback callable that wants it.

A second solution that might be a little bit more complex involves using a Message interface object that takes a given message, a given callback function, and figures out (by examining the callback function and message signal) exactly how these two objects should interact ('what sort of information does the callback need passed to it? is that information in the message?'). It would be potentially very complex, but also potentially avoids decorator dependencies. It might mean doing things like creating a dictionary that associates certain message signals with certain parameters (IE, if you have a signal of 'mouse_move', the message MUST also have mouse_position sent to it as an extra parameter), error-catching cases where a message request failed to provide these parameters. Since I intend for all signals to be defined as constants (strings), this wouldn't be a huge deal, but it might make adding new signals a little tedious (particularly if they require extra data).

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6990
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: GUI interface writ in Python

Postby The Great Hippo » Sun Aug 10, 2014 7:29 pm UTC

Here is my Registry object which I am using for my Dispatchers. The Registry object is essentially a dictionary that holds a signal : set-of-callable mapping; when a message with signal x arrives, it is directed to the Registry, where we send it to all the callables associated with signal x.

The interesting part for me was the error catching: I want to ensure that every callable you associate with a signal accepts one (and only one) argument; the message itself. However, there's a problem: In python, bound functions (IE, methods) automatically accept themselves as the initial argument (class Foo has method foo; method foo accepts itself as its first argument). Which means in the case of bound functions, I need to ensure the callable accepts two arguments -- itself, and the message.

I did some rummaging and solved this problem with the notify module:

Code: Select all

import notify

ALL_SIGNALS = 'ALL SIGNALS' # For cases where I want a set of callables to receive all signals.

def is_hashable(value):
    """is_hashable(value) -> True or False

    Simple test to determine if a value is hashable (to avoid non-hashable
    objects being assigned as signals and/or senders).
    """
    try:
        hash(value)
    except TypeError:
        return False
    return True

def is_bound(callable_):
    """is_bound(func) -> True or False

    Determine if a given callable is bound to a class (IE, it's a method)."""
    return callable_.__self__ is not None

class Registry(dict):
    """Simple registry dictionary for Dispatcher objects. Automates some error
    catching and makes adding new entries a little simpler.
    """
    def add(self, signal, callable_):
        """Registry.add(signal, callable_) -> None

        Check to see if the signal and callable are valid, then either add
        the callable to the signal's set, or create a new set for that signal
        (with the callable as the lone element of that set).
        """
        self.__error_checking(signal, callable_)
        if signal is None: signal = ALL_SIGNALS
        if signal in self:
            self[signal].add(callable_)
        else:
            self[signal] = set([callable_])
    def __error_checking(self, signal, callable_):
        if not is_hashable(signal):
            raise DispatcherError('All signals must be hashable.')
        if not callable(callable_):
            raise DispatcherError('Registered object must be callable.')
        argspecs = inspect.getargspec(callable_)
        if len(argspecs[0]) != 1 and not is_bound(callable_):
            raise DispatcherError('Registered object must accept one argument.')
        if len(argspecs[0]) != 2 and is_bound(callable_):
            raise DispatcherError('Registered object must accept one argument.')
I'm rather happy with this; it means a Dispatcher won't be able to register a callable that can't accept a message. Of course, that doesn't stop it from accepting a callable that accepts messages, but then proceeds to treat them like something else (like integers), but that's just one of the pitfalls of ducktype language, I guess.

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6990
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: GUI interface writ in Python

Postby The Great Hippo » Wed Aug 13, 2014 12:07 am UTC

The code below is for Callbacks; an object that wraps a callable (function, method, or instance of class with __call__) and a set of arguments (positional and keyword), determines whether or not all the expected positional arguments / expected keywords are present (and none are unexpected), and raises an exception if not.

The Callback also accepts a metakey keyword; a list or set of strings that correspond to values which we don't yet have, but expect to have when the Callback fires. The error checking code checks to make sure those keys are expected / not unexpected, too.
Spoiler:

Code: Select all

"""Contains the Callback class, which instantiates an object that will wrap a
given callable -- and the arguments you want to pass to it (positional and
keyword) -- performing error checking to ensure that the arguments you're
eventually going to pass to it are valid. Additionally, you may define any
keywords you want to pass to the callable at run-time under the 'metakeys'
keyword, allowing it to error-check for keywords before you've determined what
their values are.
"""

from callbacks.__init__ import *
from callbacks.errors import CallbackError

class Callback:
    def __init__(self, callable_, *args, **kwargs):
        """Callback.__init__(callable_, *args, **kwargs) -> Callback

        Initiate a Callback object. This object wraps a callable and its
        arguments, performing error checking and also providing details about
        the callable for the CallbackCapable object.

        NOTE: 'metakeys' is a reserved keyword. Any callable you wrap that takes
        'metakeys' as a keyword argument won't get it, because we use it here to
        store extra information about the callable when we wrap it!

        Keyword Arguments:
            metakeys - A set or list of strings that correspond to keys we
            expect to receive when a callable is finally called.

        Attributes:
            callable_ - The callable object (function, method, or instance of
            a class with __call__).
            bound_to - The instance this callable is bound to. If the callable
            is not bound to an instance, will be set to 'None'.
            args - The positional arguments to pass to the callable.
            kwargs - The keyword arguments to pass to the callable.
            metakeys - See 'Keyword Arguments'.
        """
        self.callable_ = callable_
        self.args = args
        self.kwargs = kwargs
        self.metakeys = self.kwargs.pop('metakeys', None)
        try:
            self.bound_to = self.callable_.__self__
        except AttributeError:
            self.bound_to = None
        CallbackError.error_checking(self)
    def __call__(self, extra_keywords = {}):
        if extra_keywords is {}:
            self.callable_(*self.args, **self.kwargs)
        else:
            # metakey work goes here
            pass


class Foo:
    def foo(self, message):
        print (message)

def foo(a, b, *, key_n, key_d = 5, key_b, key_a = None):
    print (a)
    print (b)
    print (key_n)
    print (key_d)
    print (key_b)
    print (key_a)


f = Foo()
That was the easy part; next is the bit that does the actual error-checking:

Code: Select all

"""Error types for Callbacks."""

import inspect

def __getfullargspec(callable_):
    # Wrap inspect.getfullargspec so it can handle instances of classes with
    # __call__. Raise exception if you pass a class to it.
    if inspect.isclass(callable_):
        raise TypeError('%r is not a Python function' % callable_)
    try:
        return inspect.getfullargspec(callable_)
    except TypeError:
        if hasattr(callable_, '__call__'):
            return inspect.getfullargspec(callable_.__call__)
        else:
            raise TypeError('%r is not a Python function' % callable_)

__getfullargspec.__doc__ = inspect.getfullargspec.__doc__

##############################
# Functions for type-checking.
##############################

def is_iterable(obj):
    try:
        iter(obj)
    except TypeError:
        return False
    return True

def is_string(obj):
    return isinstance(obj, str)

is_class = inspect.isclass

##############################
# Functions for positional argument checking.
##############################

def pos_arg_check(callable_, args, bound_to = None):
    """pos_arg_check(callable_, args, is_bound = None) -> (min_args, max_args,
    off_by)

    Check the minimum and maximum number of arguments callable_ requires. Return
    both these values and the extent to which the supplied arguments extends
    beyond them (as a tuple).
    """
    number_of_args = len(args)
    off_by = 0
    argspec = __getfullargspec(callable_)
    max_args = len(argspec[0])
    if bound_to: # If this is a bound method, disregard the 'self' parameter
        max_args -= 1
    min_args = max_args
    if argspec[3]:
        min_args -= len(argspec[3]) # Subtract arguments with default values
    if number_of_args < min_args:
        off_by = number_of_args - min_args
    if number_of_args > max_args:
        off_by = number_of_args - max_args
    return (min_args, max_args, off_by)


def key_arg_check(callable_, kwargs, metakeys = None):
    """key_arg_check(callable_, kwargs) -> (missing_keys, extra_keys)

    Check to see if all the keys a callable expects are present in kwargs, and
    none of the keys kwargs contains are unexpected by the callable. 'Metakeys'
    are keys for which no value yet exists, but will eventually be included in
    kwargs. Return a tuple containing keys that are missing and keys that were
    not expected.
    """
    if metakeys is None: metakeys = []
    else: metakeys = list(metakeys)
    available_keys = list(kwargs.keys()) + metakeys
    missing_keys = set()
    extra_keys = set()
    argspec = __getfullargspec(callable_)
    min_keys = [key for key in argspec[4] if key not in argspec[5].keys()]
    all_keys = argspec[4]
    for key in available_keys:
        if key not in all_keys:
            extra_keys.add(key)
    for key in min_keys:
        if key not in available_keys:
            missing_keys.add(key)
    return (missing_keys, extra_keys)

class CallbackError(Exception):
    """Base class for Callback Errors."""
    def __init__(self, message, callback):
        Exception.__init__(self, message)

    @staticmethod
    def error_checking(callback):
        """CallbackError.error_checking(callback) -> None

        Perform basic error-checking on a given callback object, raising
        exceptions as appropriate.
        """
        CallbackError.type_checking(callback)
        CallbackError.positional_checking(callback)
        CallbackError.keyword_checking(callback)

    @staticmethod
    def type_checking(callback):
        """CallbackError.type_checking(callback) -> None

        Perform basic type-checking on a callback, ensuring that its callable
        is not a class, *is* callable, and is not a callback itself. Also ensure
        that the metakeys are iterable, and not a string.
        """
        if is_class(callback.callable_):
            raise CallbackTypeError('Callback cannot wrap class',
                                    callback.callable_, callback)
        if not callable(callback.callable_):
            raise CallbackTypeError('Callback must wrap callable object',
                                    callback.callable_, callback)
        if isinstance(callback.callable_, callback.__class__):
            raise CallbackTypeError('Callback cannot wrap callback',
                                    callback.callable_, callback)
        if callback.metakeys is not None:
            if not is_iterable(callback.metakeys):
                raise CallbackTypeError('metakeys must be iterable',
                                        callback.metakeys, callback)
            if is_string(callback.metakeys):
                raise CallbackTypeError('metakeys cannot be string',
                                        callback.metakeys, callback)
    @staticmethod
    def positional_checking(callback):
        """CallbackError.positional_checking(callback) -> None

        Ensure that a given callback does not pass too few or too many
        arguments to its callable (BEFORE we try calling it!).
        """
        arg_data = pos_arg_check(callback.callable_, callback.args,
                                callback.bound_to)
        min_args, max_args, off_by = arg_data
        if off_by != 0:
            raise CallbackPositionalArgumentError(min_args, max_args, off_by,
                                                    callback)
    @staticmethod
    def keyword_checking(callback):
        """CallbackError.keyword_checking(callback) -> None

        Ensure that a given callback does not pass unexpected keys or fail to
        provide expected keys to its callable (BEFORE we try calling it!). Also
        checks to make sure metakeys are present (if any).
        """
        key_data = key_arg_check(callback.callable_, callback.kwargs,
                                callback.metakeys)
        missing_keys, extra_keys = key_data
        if missing_keys != set() or extra_keys != set():
            raise CallbackKeywordArgumentError(missing_keys, extra_keys,
                                                callback)

class CallbackTypeError(CallbackError):
    """Base class for Callback Type Errors."""
    def __init__(self, message, value, callback):
        CallbackError.__init__(self, message, callback)

class CallbackPositionalArgumentError(CallbackError):
    def __init__(self, min_args, max_args, off_by, callback):
        if off_by < 0:
            message = 'Callable needs %r argument(s), received %r' % (min_args,
                        len(callback.args))
        if off_by > 0:
            message = 'Callable takes %r argument(s), received %r' % (max_args,
                        len(callback.args))
        CallbackError.__init__(self, message, callback)

class CallbackKeywordArgumentError(CallbackError):
    def __init__(self, missing_keys, extra_keys, callback):
        if missing_keys != set() and extra_keys != set():
            message = 'Callable requires keyword arguments: %r and does not \
accept keyword arguments: %r' % (missing_keys, extra_keys)
        if missing_keys != set() and extra_keys == set():
            message = 'Callable requires keyword arguments: %r' % (missing_keys)
        if missing_keys == set() and extra_keys != set():
            message = 'Callable does not accept keyword arguments: %r' % (
                        extra_keys)
        CallbackError.__init__(self, message, callback)
It's a bit messy, but I'm very pleased with it -- I've yet to find a case it doesn't catch (it even catches cases where you try to pass another callback into a callback -- easy to do, since callbacks have a __call__!).

It accounts for Python 3's new syntax regarding keyword arguments (foo(value, *, keyword)), can handle just about any callable, and allows me to have callbacks that actually evaluate the contents of a message that triggers them (without interfering with callbacks that don't care!). I even had to extend one of python's functions (getallargspec) to account for the fact that it doesn't work with the instances of classes (for some reason -- probably to avoid confusion with people invoking it on classes, instead of instances of classes?).

I stored the actual error checking code in the CallbackError object itself (as static classes), mostly because I didn't know where else to put it -- I didn't want to stick it in the Callback class, because even though Callback uses it, I'd rather it be under the hood -- I thought of using name-mangling, but figured this way it's where it should be and I can just import ONE error class rather than all of them (since it invokes its own subclasses).

Anyway, I'm feeling a bit full of myself at the moment; if someone wants to rain on my parade, please feel free -- I'd love to know if there are any (reasonable!) points at which my code breaks.

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6990
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: GUI interface writ in Python

Postby The Great Hippo » Thu Aug 14, 2014 3:38 pm UTC

It really bothered me that I had to pass a metakey keyword to my Callbacks. It also bothered me that there wasn't an easy way to handle passing positional arguments to the callback at call-time. Analyzing the problem netted me a much more efficient / read-friendly way of doing it: ReservedArgs, or 'Unknown'.

Spoiler:

Code: Select all

"""Contains the ReservedVal and CallBinding classes. ReservedVal instantiates
an object that reserves either a position (in a positional argument) or the
value of a keyword (in a keyword argument) for CallBinding, 'promising' that
these values will be supplied when CallBinding is called. CallBinding
instantiates a binding between a callable and arguments, allowing us to call
the binding later.
"""

import callbacks.validate


class ReservedVal:
    def __init__(self):
        """ReservedVal() -> ReservedVal

        Instantiate a ReservedVal object. This object is a placeholder for
        argument values in CallBindings that we expect to receive when
        the CallBinding instance is called. You can set a ReservedVal as a
        positional argument or as a value for a keyword argument.
        """
        pass

Unknown = ReservedVal()

class CallBinding:
    def __init__(self, callable_, *args, **kwargs):
        """CallBinding(callable_, *args, **kwargs) -> CallBinding

        Instantiate a CallBinding object. This object is a binding between
        a callable and its arguments, which can be called later. It supports
        positional and keyword arguments and argument validation. With
        ReservedVal, it also supports argument bindings in cases where we do not
        yet know the value of the argument -- but expect to receive it when the
        binding is finally called.

        Attributes:
            callable_ - A function, method, or instance of a class with
            __call__.
            bound_to - The instance the callable_ is bound to. If unbound, is
            None.
            args - The positional arguments of this binding.
            kwargs - The keyword arguments of this binding.
        """
        self.callable_ = callable_
        self.args = args
        self.kwargs = kwargs
        try:
            self.bound_to = self.callable_.__self__
        except AttributeError:
            self.bound_to = None
        callbacks.validate.validate_callbinding(self)
    def __call__(self, *replacement_args, **replacement_keys):
        final_args = self._swap_out_args(replacement_args)
        final_keys = self._swap_out_keys(replacement_keys)
        self.callable_(*final_args, **final_keys)
    def _swap_out_args(self, args):
        replacement_args = list(args)
        final_args = list()
        for arg in self.args:
            if arg is Unknown:
                final_args.append(replacement_args.pop(0))
            else:
                final_args.append(arg)
        return final_args
    def _swap_out_keys(self, kwargs):
        final_keys = {}
        for key in self.kwargs:
            if self.kwargs[key] is Unknown:
                final_keys[key] = kwargs[key]
            else:
                final_keys[key] = self.kwargs[key]
        return final_keys





def foo(x, *, key = None, a = 0):
    print (x)
    print (key)

cb = CallBinding(foo, Unknown, key = Unknown)
I also shifted all the error checking code over to 'validate.py' -- it's now actually much easier to work with (and, in fact, much more portable!) since I don't need to deal with the metakey anymore.

Essentially, ReservedVal -- or 'Unknown' -- is a singleton that, when you pass it as an argument or the value of a keyword to a CallBinding (oh yes, I renamed them 'CallBindings', because I realized that's essentially what they are), serves to signal that this is a value we'll receive when the callbinding is called. At call-time, we swap out any value set to 'Unknown' with whatever values we passed to the CallBinding when we called it.

This is much, much easier. Plus, I realized that with a little work on my ReservedVal object, I can allow you to do neat stuff like bind an argument in a call that's 'Unknown + 5' -- which means we don't know what the value is yet, but we know that when we get it, we'll need to add 5 to it.

====

All the above being said, I realize that this approach relies on a singleton object, 'Unknown'. I was wondering if anyone might know one or two things -- if there's a better name than 'Unknown' (is this essentially an Algebraic Type?), and what method I should use to create a simple singleton. I thought about just loading an unknown.py module, but if I'm going to support operations on Unknowns, that won't work. Something simple -- like creating the Unknown object instantiated in the __init__.py and loading it everywhere -- might work?

Breakfast
Posts: 117
Joined: Tue Jun 16, 2009 7:34 pm UTC
Location: Coming to a table near you

Re: GUI interface writ in Python

Postby Breakfast » Thu Aug 14, 2014 3:56 pm UTC

I'd like to let you know that I'm still reading everything you post! I just haven't had time to digest it or comment on it.

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6990
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: GUI interface writ in Python

Postby The Great Hippo » Thu Aug 14, 2014 5:39 pm UTC

No problem; I appreciate it!

I just finished making some more modifications and error-testing; everything works very nicely. You can, for instance, have foo:

Code: Select all

def foo(x, y, z, *, k1, k2):
    print (x)
    print (y)
    print (z)
    print (k1)
    print (k2)


cb = CallBinding(foo, 3, Unknown, 5, k1 = 'hello', k2 = Unknown)
And type in the following:

Code: Select all

>> cb(10, k2 = 'goodbye')
Which will produce the following output:

Code: Select all

3
10
5
hello
goodbye
The important thing being, when you instantiate the CallBinding object, it checks to ensure that foo is 1) A non-class, callable object, and 2) will accept all the parameters you're feeding it. When you call foo through the CallBinding (via cb()), it takes any additional parameters you feed to it, checks to make sure you gave it enough extra parameters to replace all of its Unknowns, and then is called with those extra parameters in place of the Uknowns.

An optional method, 'CallBinding.truncated_call()', works just the same -- except if you feed it keyword arguments the callable it's wrapping doesn't accept, it'll just ignore them and pass the acceptable ones to the function. This is so I can do things like shove an object's __dict__ into a callbinding when I know it'll have the right attributes, but I really don't feel like picking those attributes out of it to send to the callbinding.

I'm debating if the next step should be to add arithmetic to the Unknowns (so you can set 'Unknown + 5' as a value; that means when called, it will replace Unknown with a new value and immediately add +5) or just move the hell on to something else.

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6990
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: GUI interface writ in Python

Postby The Great Hippo » Sun Aug 17, 2014 10:23 pm UTC

So, I recently found out that Python 3.4 supports inspect.getcallargs, which raises an appropriate error if a function doesn't accept the parameters you want to supply it. Also, Python now has function signatures, which are great for inspecting parameters, binding them, and swapping them out -- all things I've been trying to do.

On one hand it's a little depressing to find out I spent two weeks solving a problem that's already been solved. On the other hand, I learned craploads of new stuff, and it's cool to realize that the problem I was working on is credible enough that someone else took time to solve it!

Plus, now I can use their much better solution in my code.

User avatar
PM 2Ring
Posts: 3632
Joined: Mon Jan 26, 2009 3:19 pm UTC
Location: Mid north coast, NSW, Australia

Re: GUI interface writ in Python

Postby PM 2Ring » Thu Aug 21, 2014 10:14 am UTC

Interesting!

I must confess I've only skimmed your code snippets here, but you seem to be putting an awful lot of work into this.

Does your GUI system sit on top of pygame? If not, how does it do the low level stuff?

Is this "just" a learning exercise, or do you have a reason why you don't want to use an existing GUI system, eg GTK? Mind you, even if it is "just" a learning exercise, it looks like an excellent one, and I'm sure that you'll be a Python expert and a GUI guru by the time you're finished. OTOH, when doing a project like this you may end up re-inventing a lot of wheels that others have already perfected, which can be a tad annoying.

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6990
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: GUI interface writ in Python

Postby The Great Hippo » Thu Aug 21, 2014 12:06 pm UTC

PM 2Ring wrote:Does your GUI system sit on top of pygame? If not, how does it do the low level stuff?
It sits on top of pygame, using it for the SDL library -- and retrieving user events (key press, mouse movement, mouse clicks, etc).
PM 2Ring wrote:Is this "just" a learning exercise, or do you have a reason why you don't want to use an existing GUI system, eg GTK? Mind you, even if it is "just" a learning exercise, it looks like an excellent one, and I'm sure that you'll be a Python expert and a GUI guru by the time you're finished. OTOH, when doing a project like this you may end up re-inventing a lot of wheels that others have already perfected, which can be a tad annoying.
Yeah, my entire experience in learning Python has consisted of variations on that theme: Discover problem X. Work on a solution for problem X, learning all about problem X along the way. Finally formulate a functional solution for problem X, only to discover that there's already a pre-made solution for problem X that works 10 times better than the one I came up with.

I'm mostly not using a pre-made GUI because I want to understand how GUIs work; I've actually picked through OcempGUI and pgu (two GUIs that sit on top of pygame) to better understand how to build a GUI in pygame. There's a certain irony in looking at the source code for perfectly functional GUIs that sit on top of pygame to build a functional GUI on top of pygame. That being said, it's definitely taught me a lot about GUIs and Python.

Just the other day, I started reading Python PEPs -- and realized I was actually understanding about 75% of them.

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6990
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: GUI interface writ in Python

Postby The Great Hippo » Fri Aug 22, 2014 6:47 pm UTC

In working to understand PEP 362, I think I might have found two errors in their example code that uses annotations for type-checking (below):
Spoiler:

Code: Select all

import inspect
import functools

def checktypes(func):
    '''Decorator to verify arguments and return types

    Example:

        >>> @checktypes
        ... def test(a:int, b:str) -> int:
        ...     return int(a * b)

        >>> test(10, '1')
        1111111111

        >>> test(10, 1)
        Traceback (most recent call last):
          ...
        ValueError: foo: wrong type of 'b' argument, 'str' expected, got 'int'
    '''

    sig = inspect.signature(func)

    types = {}
    for param in sig.parameters.values():
        # Iterate through function's parameters and build the list of
        # arguments types
        type_ = param.annotation
        if type_ is param.empty or not inspect.isclass(type_):
            # Missing annotation or not a type, skip it
            continue

        types[param.name] = type_

        # If the argument has a type specified, let's check that its
        # default value (if present) conforms with the type.
        if param.default is not param.empty and not isinstance(param.default, type_):
            raise ValueError("{func}: wrong type of a default value for {arg!r}". \
                             format(func=func.__qualname__, arg=param.name))

    def check_type(sig, arg_name, arg_type, arg_value):
        # Internal function that encapsulates arguments type checking
        if not isinstance(arg_value, arg_type):
            raise ValueError("{func}: wrong type of {arg!r} argument, " \
                             "{exp!r} expected, got {got!r}". \
                             format(func=func.__qualname__, arg=arg_name,
                                    exp=arg_type.__name__, got=type(arg_value).__name__))

    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        # Let's bind the arguments
        ba = sig.bind(*args, **kwargs)
        for arg_name, arg in ba.arguments.items():
            # And iterate through the bound arguments
            try:
                type_ = types[arg_name]
            except KeyError:
                continue
            else:
                # OK, we have a type for the argument, lets get the corresponding
                # parameter description from the signature object
                param = sig.parameters[arg_name]
                if param.kind == param.VAR_POSITIONAL:
                    # If this parameter is a variable-argument parameter,
                    # then we need to check each of its values
                    for value in arg:
                        check_type(sig, arg_name, type_, value)
                elif param.kind == param.VAR_KEYWORD:
                    # If this parameter is a variable-keyword-argument parameter:
                    for subname, value in arg.items():
                        check_type(sig, arg_name + ':' + subname, type_, value)
                else:
                    # And, finally, if this parameter a regular one:
                    check_type(sig, arg_name, type_, arg)

        result = func(*ba.args, **ba.kwargs)

        # The last bit - let's check that the result is correct
        return_type = sig.return_annotation
        if (return_type is not sig._empty and
                isinstance(return_type, type) and
                not isinstance(result, return_type)):

            raise ValueError('{func}: wrong return type, {exp} expected, got {got}'. \
                             format(func=func.__qualname__, exp=return_type.__name__,
                                    got=type(result).__name__))
        return result

    return wrapper
Specifically, sig._empty doesn't exist (it's just sig.empty) and when they define check_type (the internal check-typing function), they seem to forget to skip the check if the parameter has no annotation (so when you use it, function parameters without annotations immediately pass an exception since whatever value you pass to it won't equal 'parameter.empty').

I did some searching to see if anyone else noticed this or if I'm just crazy. I made the correction in the code I'm using and the function works just fine, now; if annotations are provided and you wrap the function with the @checktypes, it passes appropriate errors -- and if you don't provide annotations (but still wrap it with @checktypes), it doesn't raise an error for unannotated values.

On one hand, it'd be really cool if I'm starting to notice errors in other people's code, and correct them when appropriate! On the other hand, I kind of just want confirmation from an outside source that this is an error, and I am correcting it (here's my updated code below; pretty much identical, but my comments are slightly different, and contains the two minor corrections):
Spoiler:

Code: Select all

import inspect
import functools

def checktypes(func):
    """Decorator to check types of arguments and returns using annotations.
    Largely stolen from PEP 362.

    Example:

        @checktypes
        def foo(a : int, b : str) -> int:
            return int(a * b)

        >>>foo(10, '1')
        1111111111

        >>>foo(10, 1):
        Traceback (most recent call last):
          ...
        ValueError: foo: wrong type of 'b' argument, 'str' expected, got 'int'
    """
    sig = inspect.signature(func)

    types = {}
    for param in sig.parameters.values():
        # Iterate through function's parameters, building the list of argument
        # types.
        type_ = param.annotation
        if type is param.empty or not inspect.isclass(type_):
            # Missing annotation or is not a type; skip it.
            continue
        types[param.name] = type_

        # If the argument has a type specified, let's check that its
        # default value (if present) conforms with the type.
        if param.default is not param.empty and not isinstance(param.default, type_):
            raise ValueError("{func}: wrong type of a default value for {arg!r}". \
                            format(func=func.__qualname__, arg=param.name))
    def check_type(sig, arg_name, arg_type, arg_value):
        # Internal function that encapsulates arguments type checking
        if not (isinstance(arg_value, arg_type)) and arg_type != sig.empty:
            raise ValueError("{func}: wrong type of {arg!r} argument, " \
                            "{exp!r} expected, got {got!r}". \
                            format(func=func.__qualname__, arg=arg_name,
                                   exp=arg_type.__name__, got=type(arg_value).__name__))

    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        # Let's bind the arguments.
        ba = sig.bind(*args, **kwargs)
        for arg_name, arg in ba.arguments.items():
            # Iterate through bound arguments
            try:
                type_ = types[arg_name]
            except KeyError:
                continue
            else:
                # Okay, we have a type for the argument, lets get the corresponding
                # parameter description from the signature argument
                param = sig.parameters[arg_name]
                if param.kind == param.VAR_POSITIONAL:
                    # If this parameter is a variable-argument parameter,
                    # then we need to check each of its values
                    for value in arg:
                        check_type(sig, arg_name, type_, value)
                elif param.kind == param.VAR_KEYWORD:
                    # If this parameter is a variable-keyword-argument parameter:
                    for subname, value in arg.items():
                        check_type(sig, arg_name + ':' + subname, type_, value)
                else:
                    # And finally, if this parameter is just an ordinary one
                    check_type(sig, arg_name, type_, arg)
        result = func(*ba.args, **ba.kwargs)

        # The last bit -- let's check that the result is correct.
        return_type = sig.return_annotation
        if (return_type is not sig.empty and
            isinstance(return_type, type) and
            not isinstance(result, return_type)):

            raise ValueError('{func}: wrong return type, {exp} expected, got {got}'. \
                            format(func=func.__qualname__, exp=return_type.__name__,
                                   got=type(result).__name__))
        return result

    return wrapper


EDIT: I also just noticed a similar error when dealing with default key values, which I'm working to correct.

EDIT-EDIT: Oh, wait, crap, I just noticed a typo I made in copying the code over (type instead of type_); that's the issue; nvm. The only thing wrong then is the minor typo of 'sig._empty', and I can see how that happened (because the empty value is assigned in the inspect module as 'inspect._empty', which is referenced in the signature class under 'Signature.empty'). Once I fix that, their code works exactly as expected.

User avatar
Jplus
Posts: 1709
Joined: Wed Apr 21, 2010 12:29 pm UTC
Location: Netherlands

Re: GUI interface writ in Python

Postby Jplus » Sun Aug 24, 2014 8:28 pm UTC

You seem to be doing really well, Great Hippo! I'm impressed.

Sorry to sound like a recruiter, but I believe you could be an awesome part of the red spider team. ;-)
"There are only two hard problems in computer science: cache coherence, naming things, and off-by-one errors." (Phil Karlton and Leon Bambrick)

coding and xkcd combined

(Julian/Julian's)

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6990
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: GUI interface writ in Python

Postby The Great Hippo » Mon Aug 25, 2014 9:45 am UTC

I have no idea what the red spider team is, but I could give it a looksee! I kind of want to get somewhere on this project of mine before taking on any other sort of monster, though.

Also, I finished consolidating inspect.signature with my code, today; I now use the inspect module to generate a FuncSpec object, which can return a CallSpec object, which tells you what to expect if you call a given function/method/callable with a given group of arguments. IE, 'What will happen if I pass X to Y?', except without raising an error (yet).

Python really isn't a 'look-before-you-leap' kind of language, but I think that callbacks -- functions that you're storing to call later -- are a rare case where you're going to want to do some heavy introspection beforehand -- since otherwise, any function errors won't show up until you call the function (and you might not call it for a good, long while!).
Spoiler:

Code: Select all

import inspect
import collections

EMPTY = inspect.Signature.empty # For cases where there is no value (Not even None!)

class FuncSpec:

    def __init__(self, func):
        """FuncSpec(func) -> FuncSpec

        Instantiate FuncSpec, an object that calculates details regarding
        a function (ANY callable) and the arguments it will accept. When
        invoked on a class, it looks at __init__; when invoked on an
        instance of a class, it looks at __call__.

        Attributes:
            self.infinite - Special class-internal object for handling
            callables with *args and **kwargs (infinite allowable positional
            and keyword arguments).
            self.func - The callable we are examining.
            self.sig - The signature of the callable.
            self.parameters - The parameters of the signature.
        """
        self.infinite = FuncSpec._infinite
        self.func = func
        self.sig = inspect.signature(self.func)
        self.parameters = self.sig.parameters
        max_args, min_args = 0, 0
        max_kwargs, min_kwargs = set(), set()
        for param in self.parameters.values():
            if param.kind == param.VAR_POSITIONAL:
                max_args = self.infinite
            if param.kind == param.VAR_KEYWORD:
                max_kwargs = self.infinite
            if param.kind == param.POSITIONAL_OR_KEYWORD or param.POSITIONAL_ONLY:
                max_args += 1
                if param.default == EMPTY:
                    min_args += 1
            if param.kind == param.KEYWORD_ONLY:
                max_kwargs.add(param.name)
                if param.default == EMPTY:
                    min_kwargs.add(param.name)
        self.max_args, self.min_args = max_args, min_args
        self.max_kwargs, self.min_kwargs = max_kwargs, min_kwargs
    def callspec(self, *args, **kwargs):
        """FuncSpec.callspec(*args, **kwargs) -> CallSpec

        Return a CallSpec instance (a class internal to FuncSpec)
        that provides detailed information regarding what would happen
        if the callable was called with the given arguments.
        """
        return FuncSpec.CallSpec(self, *args, **kwargs)

    class CallSpec:
        def __init__(self, funcspec, *args, **kwargs):
            """FuncSpec.CallSpec(funcspec, *args, **kwargs) -> CallSpec

            Instantiate a CallSpec object. Provides detailed information
            regarding what will happen if you call a given callable
            with the provided arguments.

            Attributes:
                binding - An instance of FuncSpec.TestBinding; an alternative
                to inspect.signature.bind that raises no exceptions. Used
                to bind arguments to parameters.
                extra_args - The number of positional arguments that are
                not expected.
                missing_args - The number of positional arguments that are
                expected, but not present.
                extra_kwargs - A set of every keyword argument that is
                not expected.
                missing_kwargs - A set of every keyword argument that is
                expected, but not present.
                funcspec - The FuncSpec instance that generated this CallSpec.
                exception - The exception expected to be raised if you call
                the callable with these arguments. If None, no exception is
                expected.
                """
            binding = FuncSpec.TestBinding(funcspec, *args, **kwargs)
            extra_args = binding.unbound_args
            extra_kwargs = binding.unbound_kwargs
            missing_args = funcspec.min_args - binding.bound_args
            if missing_args < 0: missing_args = 0
            missing_kwargs = {k for k in funcspec.min_kwargs if k not in binding.bound_kwargs}
            self.extra_args, self.extra_kwargs = extra_args, extra_kwargs
            self.missing_args, self.missing_kwargs = missing_args, missing_kwargs
            self.funcspec, self.binding = funcspec, binding
            self.exception = self.get_exception()
        def get_exception(self):
            funcspec = self.funcspec
            func = funcspec.func
            try:
                func_name = func.__name__
            except AttributeError:
                func_name = func.__class__.__name__ + '.' + func.__call__.__name__
            if self.extra_args != 0 or self.missing_args != 0:
                received_args = self.binding.bound_args + self.binding.unbound_args
                if funcspec.max_args == funcspec.min_args:
                    message = '%s() expected %s positional argument(s); received %s' % (func_name, funcspec.max_args,
                                                                             received_args)
                else:
                    message = '%s() expected %s to %s positional argument(s); received %s' % (func_name,
                                                                            funcspec.min_args, funcspec.max_args,
                                                                            received_args)
                return TypeError(message)
            if self.extra_kwargs != set() or self.missing_kwargs != set():
                if self.extra_kwargs != set() and self.missing_kwargs == set():
                    message = '%s() received unexpected keyword argument(s): %r' % (func_name, self.extra_kwargs)
                if self.extra_kwargs == set() and self.missing_kwargs != set():
                    message = '%s() did not receive expected keyword argument(s): %r' % (func_name, self.missing_kwargs)
                if self.extra_kwargs != set() and self.missing_kwargs != set():
                    message = ('%s() received unexpected keyword argument(s): %r ' % (func_name, self.extra_kwargs) +
                               'and did not receive keyword argument(s): %r' % (self.missing_kwargs))
                return TypeError(message)

               
    class TestBinding(collections.OrderedDict):
        def __init__(self, funcspec, *args, **kwargs):
            """FuncSpec.TestBinding(funcspec, *args, **kwargs) -> TestBinding

            Instantiate a TestBinding object. A 'softer' variant of
            inspect.signature.binding; binds arguments to a callable, but
            without raising exceptions.

            Attributes:
                funcspec - The FuncSpec this binding is attached to.
                bound_args - The number of arguments successfully bound.
                unbound_args - The number of arguments not successfully bound.
                bound_kwargs - A set of all the keywords that were successfully
                bound.
                unbound_kwargs - A set of all the keywords that were not
                successfully bound.
                default_args - The number arguments that were bound to their
                function's default values.
                default_kwargs - A set of all the keywords that were bound to
                their function's default values.
            """
            collections.OrderedDict.__init__(self)
            params = funcspec.parameters
            i, bound_args, bound_kwargs = -1, 0, set()
            unbound_args, unbound_kwargs = 0, set()
            default_args, default_kwargs = 0, set()
            for param in params.values():
                i += 1
                if param.kind == param.VAR_POSITIONAL:
                    self[param.name] = args[i:]
                    bound_args += len(self[param.name])
                if (param.kind == param.POSITIONAL_OR_KEYWORD or
                    param.kind == param.POSITIONAL_ONLY):
                    try:
                        self[param.name] = args[i]
                        bound_args += 1
                    except IndexError: # We're out of args
                        if param.name in kwargs: # Search kwargs
                            self[param.name] = kwargs[param.name]
                            bound_args += 1
                        else: # Not found, set to default or EMPTY
                            self[param.name] = param.default
                            if param.default != EMPTY:
                                default_args += 1
                if param.kind == param.KEYWORD_ONLY:
                    try:
                        self[param.name] = kwargs[param.name]
                        bound_kwargs.add(param.name)
                    except KeyError: # Not found in kwargs, set to default or EMPTY
                        self[param.name] = param.default
                        if param.default != EMPTY:
                            default_kwargs.add(param.name)
                if param.kind == param.VAR_KEYWORD:
                    self[param.name] = {k : v for (k, v) in kwargs.items() if k not in self}
                    bound_kwargs.update({k for k in self[param.name].keys()})
            if funcspec.max_args < len(args):
                unbound_args = len(args) - funcspec.max_args
            unbound_kwargs = {k for k in kwargs.keys() if k not in bound_kwargs and k not in self.keys()}
            self.bound_args, self.bound_kwargs = bound_args, bound_kwargs
            self.unbound_args, self.unbound_kwargs = unbound_args, unbound_kwargs
            self.default_args, self.default_kwargs = default_args, default_kwargs

    class _Infinite:
        """For internal use by FuncSpec; handles cases where
        max_args or max_kwargs should be infinite because of
        *args or **kwargs.
        """
        def __lt__(self, other):
            return False
        def __le__(self, other):
            if isinstance(other, _Infinite): return True
            else: return False
        def __gt__(self, other):
            if isinstance(other, _Infinite): return False
            else: return True
        def __ge__(self, other):
            if isinstance(other, _Infinite): return True
            else: return False
        def __add__(self, other):
            return self
        def __sub__(self, other):
            return self
        def __mul__(self, other):
            return self
        def __contains__(self, other):
            return True
        def __str__(self):
            return 'infinite'
        def __repr__(self):
            return self.__class__.__name__
        def add(self, other):
            """For cases where _Infinite is standing in for a set."""
            return
        def discard(self, other):
            """For cases where _Infinite is standing in for a set."""
            return

    _infinite = _Infinite() # No touchey!

I originally included type-checking via function annotations, but I've since taken it out because -- well, I'm not using function annotations. I might re-insert them later, though.

Anyway, here's my first test:

Code: Select all

def foo(x):
    pass

f = FuncSpec(foo)
cs = f.callspec(5, 3)

Which outputs:

Code: Select all

extra args:         1
missing args:       0
extra keys:         set()
missing keys:       set()
expected exception: foo() expected 1 positional argument(s); received 2
Change the function's arguments...

Code: Select all

def foo(x, y, z = None):
    pass

f = FuncSpec(foo)
cs = f.callspec(5, 3)
Output:

Code: Select all

extra args:         0
missing args:       0
extra keys:         set()
missing keys:       set()
expected exception: None
And one more, just to demonstrate that it can handle really complex crap:

Code: Select all

def foo(x, y, z = None, *args, key_1 = 1, key_2, key_3):
    pass

f = FuncSpec(foo)
cs = f.callspec(key_4 = 9)
Output:

Code: Select all

extra args:         0
missing args:       2
extra keys:         {'key_4'}
missing keys:       {'key_2', 'key_3'}
expected exception: foo() expected 2 to infinite positional argument(s); received 0
(Rather than trying to store ALL the exceptions a function call will raise, I just decided to store the first one I expect it will raise -- since the point is to raise errors immediately anyway).

I've tested it on a bunch of different argument configurations, and it works in every case -- even stuff like assigning your positionals as keywords won't fool it. The only thing I haven't tested it on is cases where Python's built-ins have positional only arguments (rather than positional-or-keyword arguments), but I kinda suspect I will never be using Python built-ins as callbacks anyway.

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6990
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: GUI interface writ in Python

Postby The Great Hippo » Sun Aug 31, 2014 12:57 am UTC

I finally went ahead and put my funcspec module up on a repository just because I think it might actually be useful, although there's a lot of stuff I need to iron out with it (for example, I need to get the test binding to a state where you can just send it off to a function!).

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6990
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: GUI interface writ in Python

Postby The Great Hippo » Mon Sep 01, 2014 5:31 pm UTC

I have a group of callbacks (functions I want to call later) associated with a signal (a type of message object) and a sender (the object that sent the message object). The callbacks are bound to values I want passed to them upon being called. When the right message from the right sender arrives, the associated callbacks are called with these bound values.

The problem is that not all of these bound values are ACTUALLY the values I want passed. Because sometimes, I don't actually know what value I want to pass to my callback *until* the message arrives (which should have the value I need). That's fine and dandy -- but until now -- I haven't found a way to guarantee that the message object will always have the values a callback needs, nor a way to tell the callback where to look on the message object *for* those values. To make matters worse, I want to error-check my callbacks at creation -- *before* I even call them!

My solution, right now, is this:

Code: Select all

class _:
   def __init__(self, attr):
      self.attr = attr
   def __call__(self, instance):
      return getattr(instance, self.attr)

class Message:
   def __init__(self, sender):
      self.sender = sender

onMessage = Message(sender = _('sender'))

class Mouse(Message):
   def __init__(self, sender, xy):
      Message.__init__(self, sender)
      self.xy = xy

onMouse = Mouse(sender = _('sender'), xy = _('xy'))

class MouseClick(Mouse):
   def __init__(self, sender, xy, button):
      Mouse.__init__(sender, xy)
      self.button = button

onMouseClick(sender = _('sender'), xy = _('xy'), button = _('button'))

class MouseDrag(Mouse):
   def __init__(self, sender, xy, rel):
      Mouse.__init__(self, sender, xy)
      self.rel = rel # 2D vector of how much the mouse has moved

onMouseDrag = MouseDrag(sender = _('sender'), xy = _('xy'), rel = _('rel'))



So, when I create a callback that lets you drag a window around by its dragbar...

Code: Select all

Window.create_callback(signal = onMouseDrag, sender = Dragbar, callback = Window.Move, args = onMouseDrag.rel)


...I can get the message instance's rel attribute with...

Code: Select all

onMouseDrag.rel(message_instance)


My only apprehension right now is that 1) It's a little messy (sucks having to instantiate each class shortly after defining it!), and 2) I wish there was an easier way to create the path back to the instance attribute rather than using the '_' class.

Luckily, I'll be able to check for the '_' class in my arguments, and if it's there, I'll know to pass the message instance to the argument to get the *actual* argument I'm passing to the callback. The other cool thing I realized about this solution: I now have hierarchies of signals, rather than independent signals. Which means I can have callbacks that fire on *any* mouse message -- or *any* message, period. Which might end up being really useful later on.

Nyktos
Posts: 138
Joined: Mon Mar 02, 2009 4:02 pm UTC

Re: GUI interface writ in Python

Postby Nyktos » Mon Sep 01, 2014 5:36 pm UTC

The Great Hippo wrote:I finally went ahead and put my funcspec module up on a repository just because I think it might actually be useful, although there's a lot of stuff I need to iron out with it (for example, I need to get the test binding to a state where you can just send it off to a function!).
Just so you're aware, if you ever do decide to try and make it run on pre-3.3 versions of Python, there is a module on PyPI that provides the equivalent of inspect.signature for older versions.

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6990
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: GUI interface writ in Python

Postby The Great Hippo » Mon Sep 01, 2014 5:46 pm UTC

Oh, hey -- cool! Thanks! Although, actually, much to my embarrassment, I recently realized my entire FuncSpec module can be replaced with the following code:

Code: Select all

import inspect

class SafeBind:
    def __init__(self, func, *args, **kwargs):
        if not callable(func):
            raise TypeError('%r is not callable' % (func))
        try:
            inspect.getcallargs(func, *args, **kwargs)
            self.exception = None
        except TypeError as e:
            self.exception = e
        self.func = func
        self.args = args
        self.kwargs = kwargs

    @classmethod
    def from_safebind(cls, other, *args, **kwargs):
        new_args = other.args + args
        new_kwargs = dict(list(other.kwargs.items()) + list(kwargs.items()))
        instance = cls(other.func, *new_args, **new_kwargs)
        return instance
Which should work in Python 2.7, too (haven't tested it yet, though!). The only thing the above can't do is provide an effective means to truncate extra keyword / positionals from my function bindings (because it doesn't tell me how 'wrong' I am; it just gives me the appropriate exception object for my wrongness).

But I don't think I'm going to be trying to truncate function bindings to 'fit' into a function anyway; I initially thought I'd have to because of how message objects would get passed to callbacks, but I recently realized it's much more effective to give the callback a path to retrieve the relevant message object instance's attribute rather than trying to throw everything in the message object instance's __dict__ at the callback (and letting God + math sort it out).

User avatar
The Great Hippo
Swans ARE SHARP
Posts: 6990
Joined: Fri Dec 14, 2007 4:43 am UTC
Location: behind you

Re: GUI interface writ in Python

Postby The Great Hippo » Sat Sep 06, 2014 11:51 pm UTC

Finally finished re-organizing, re-structuring, and consolidating all of my code; because of just how radical my changes were from a month ago, I just deleted my old adelaide depository and created a new one to hold the code. It's here, same place as the old one, except it's -- well, new.

It works! In its current state, it looks quite unusual (run from Python 3.x, you're going to get doughnuts rather than windows -- because I'm experimenting with pie menus), but I've essentially got a GUI that can be easily navigated / extended, I think. The next stage I'm working on is extending the GUI a bit and making some of the inheritance for widgets a little saner.


Return to “Coding”

Who is online

Users browsing this forum: No registered users and 9 guests