Page 1 of 1

How do you manage Python dependencies?

Posted: Mon Jul 18, 2016 3:31 am UTC
by jacques01
I have a project that has several dependencies that can be installed using pip.

I am not looking for a build tool.

I am just looking for a way that another programmer could look at some configuration file, and understand each of the dependencies + how to get them.

e.g. in my made up notation

{
"dependencies":
[
{"package":"bs4", "name":"BeautifulSoup 4", "install":"pip install bs4", "purpose":"Parse HTML"}
]
}

Re: How do you manage Python dependencies?

Posted: Mon Jul 18, 2016 4:42 am UTC
by phlip
I believe this is the traditional method.

The short version: list the requirements in a text file called requirements.txt, one per line. Then people can run "pip install -r requirements.txt" and grab everything.

Re: How do you manage Python dependencies?

Posted: Mon Jul 18, 2016 5:52 am UTC
by Xanthir
Tangent, but: don't use BeautifulSoup. Use html5lib, which actually implements the HTML parsing algorithm from the spec.

Re: How do you manage Python dependencies?

Posted: Tue Jul 19, 2016 12:21 am UTC
by jacques01
Thank you. I'll try to do that.

Wait, why not to use BeautifulSoup?

Re: How do you manage Python dependencies?

Posted: Tue Jul 19, 2016 6:08 am UTC
by Xanthir
Because, to the best of my knowledge, it doesn't implement the HTML parsing algorithm from the spec. I know the maintainer of html5lib, and they've worked on the spec itself in the past.

Re: How do you manage Python dependencies?

Posted: Thu Jul 21, 2016 9:05 pm UTC
by Tegelane
But, as far as I understand, real HTML is not written according to spec and last time I used it, beautifulsoup did just what was needed - parse non-spec HTML into usable structures.

Re: How do you manage Python dependencies?

Posted: Fri Jul 22, 2016 3:37 pm UTC
by korona
With HTML 5 the spec was adjusted to the code that is used in the wild.

Re: How do you manage Python dependencies?

Posted: Sat Jul 23, 2016 3:38 am UTC
by Xanthir
Ah, you misunderstand me. I'm talking about a spec-conformant parser; that has nothing to do with spec-conformant content. The whole point of a parser conforming to the spec is that it'll handle malformed HTML in exactly the same way that browsers do (because the spec was written to match browsers, and where they differed, browsers converged on the spec). Anything that doesn't match the spec will have its own error-handling, whatever the authors thought was "reasonable", and your result may not match what a browser would produce in some cases.

Re: How do you manage Python dependencies?

Posted: Sat Jul 23, 2016 4:02 pm UTC
by thoughtfully
BeautifulSoup also overloads a lot of operators for its functionality, which can lead to the usual confusions and difficult-to-maintain code. Kinda nice for quick one-offs or interactive interpreter sessions, but beyond that, I would try to stick to html5lib nowadaze.