How do you manage Python dependencies?

A place to discuss the implementation and style of computer programs.

Moderators: phlip, Moderators General, Prelates

jacques01
Posts: 42
Joined: Thu Oct 08, 2015 4:56 am UTC

How do you manage Python dependencies?

Postby jacques01 » Mon Jul 18, 2016 3:31 am UTC

I have a project that has several dependencies that can be installed using pip.

I am not looking for a build tool.

I am just looking for a way that another programmer could look at some configuration file, and understand each of the dependencies + how to get them.

e.g. in my made up notation

{
"dependencies":
[
{"package":"bs4", "name":"BeautifulSoup 4", "install":"pip install bs4", "purpose":"Parse HTML"}
]
}

User avatar
phlip
Restorer of Worlds
Posts: 7543
Joined: Sat Sep 23, 2006 3:56 am UTC
Location: Australia
Contact:

Re: How do you manage Python dependencies?

Postby phlip » Mon Jul 18, 2016 4:42 am UTC

I believe this is the traditional method.

The short version: list the requirements in a text file called requirements.txt, one per line. Then people can run "pip install -r requirements.txt" and grab everything.

Code: Select all

enum ಠ_ಠ {°□°╰=1, °Д°╰, ಠ益ಠ╰};
void ┻━┻︵​╰(ಠ_ಠ ⚠) {exit((int)⚠);}
[he/him/his]

User avatar
Xanthir
My HERO!!!
Posts: 5215
Joined: Tue Feb 20, 2007 12:49 am UTC
Location: The Googleplex
Contact:

Re: How do you manage Python dependencies?

Postby Xanthir » Mon Jul 18, 2016 5:52 am UTC

Tangent, but: don't use BeautifulSoup. Use html5lib, which actually implements the HTML parsing algorithm from the spec.
(defun fibs (n &optional (a 1) (b 1)) (take n (unfold '+ a b)))

jacques01
Posts: 42
Joined: Thu Oct 08, 2015 4:56 am UTC

Re: How do you manage Python dependencies?

Postby jacques01 » Tue Jul 19, 2016 12:21 am UTC

Thank you. I'll try to do that.

Wait, why not to use BeautifulSoup?

User avatar
Xanthir
My HERO!!!
Posts: 5215
Joined: Tue Feb 20, 2007 12:49 am UTC
Location: The Googleplex
Contact:

Re: How do you manage Python dependencies?

Postby Xanthir » Tue Jul 19, 2016 6:08 am UTC

Because, to the best of my knowledge, it doesn't implement the HTML parsing algorithm from the spec. I know the maintainer of html5lib, and they've worked on the spec itself in the past.
(defun fibs (n &optional (a 1) (b 1)) (take n (unfold '+ a b)))

Tegelane
Posts: 6
Joined: Mon Apr 21, 2008 7:59 am UTC

Re: How do you manage Python dependencies?

Postby Tegelane » Thu Jul 21, 2016 9:05 pm UTC

But, as far as I understand, real HTML is not written according to spec and last time I used it, beautifulsoup did just what was needed - parse non-spec HTML into usable structures.

korona
Posts: 495
Joined: Sun Jul 04, 2010 8:40 pm UTC

Re: How do you manage Python dependencies?

Postby korona » Fri Jul 22, 2016 3:37 pm UTC

With HTML 5 the spec was adjusted to the code that is used in the wild.

User avatar
Xanthir
My HERO!!!
Posts: 5215
Joined: Tue Feb 20, 2007 12:49 am UTC
Location: The Googleplex
Contact:

Re: How do you manage Python dependencies?

Postby Xanthir » Sat Jul 23, 2016 3:38 am UTC

Ah, you misunderstand me. I'm talking about a spec-conformant parser; that has nothing to do with spec-conformant content. The whole point of a parser conforming to the spec is that it'll handle malformed HTML in exactly the same way that browsers do (because the spec was written to match browsers, and where they differed, browsers converged on the spec). Anything that doesn't match the spec will have its own error-handling, whatever the authors thought was "reasonable", and your result may not match what a browser would produce in some cases.
(defun fibs (n &optional (a 1) (b 1)) (take n (unfold '+ a b)))

User avatar
thoughtfully
Posts: 2243
Joined: Thu Nov 01, 2007 12:25 am UTC
Location: Minneapolis, MN
Contact:

Re: How do you manage Python dependencies?

Postby thoughtfully » Sat Jul 23, 2016 4:02 pm UTC

BeautifulSoup also overloads a lot of operators for its functionality, which can lead to the usual confusions and difficult-to-maintain code. Kinda nice for quick one-offs or interactive interpreter sessions, but beyond that, I would try to stick to html5lib nowadaze.
Image
Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.
-- Antoine de Saint-Exupery


Return to “Coding”

Who is online

Users browsing this forum: No registered users and 6 guests