Rectangle 27 1474

Python 2 2.7.8 and Python 3 3.3

Good news! Python 3.4 (released March 2014) and Python 2.7.9 (released December 2014) ship with Pip. This is the best feature of any Python release. It makes the community's wealth of libraries accessible to everyone. Newbies are no longer excluded from using community libraries by the prohibitive difficulty of setup. In shipping with a package manager, Python joins Ruby, Node.js, Haskell, Perl, Go--almost every other contemporary language with a majority open-source community. Thank you Python.

And, alas for everyone using Python 2.7.8 or earlier (a sizable portion of the community). There's no plan to ship Pip to you. Manual instructions follow.

Flying in the face of its 'batteries included' motto, Python ships without a package manager. To make matters worse, Pip was--until recently--ironically difficult to install.

Download get-pip.py, being careful to save it as a .py file rather than .txt. Then, run it from the command prompt:

python get-pip.py

You possibly need an administrator command prompt to do this. Follow Start a Command Prompt as an Administrator (Microsoft TechNet).

This installs the pip package, which (in Windows) contains ...\Scripts\pip.exe that path must be in PATH environment variable to use pip from the command line (see second part of 'Alternative Instructions' for adding it to your PATH,

The official documentation tells users to install Pip and each of its dependencies from source. That's tedious for the experienced, and prohibitively difficult for newbies.

For our sake, Christoph Gohlke prepares Windows installers (.msi) for popular Python packages. He builds installers for all Python versions, both 32 and 64 bit. You need to:

For me, this installed Pip at C:\Python27\Scripts\pip.exe. Find pip.exe on your computer, then add its folder (for example, C:\Python27\Scripts) to your path (Start / Edit environment variables). Now you should be able to run pip from the command line. Try installing a package:

pip install httpie

There you go (hopefully)! Solutions for common problems are given below:

If you work in an office, you might be behind a HTTP proxy. If so, set the environment variables http_proxy and https_proxy. Most Python applications (and other free software) respect these. Example syntax:

http://proxy_url:port
http://username:password@proxy_url:port

If you're really unlucky, your proxy might be a Microsoft NTLM proxy. Free software can't cope. The only solution is to install a free software friendly proxy that forwards to the nasty proxy. http://cntlm.sourceforge.net/

Python modules can be part written in C or C++. Pip tries to compile from source. If you don't have a C/C++ compiler installed and configured, you'll see this cryptic error message.

You can fix that by installing a C++ compiler such as MinGW or Visual C++. Microsoft actually ship one specifically for use with Python. Or try Microsoft Visual C++ Compiler for Python 2.7.

Is that "can be fixed and will not be fixed" or "cannot be fixed and will not be fixed"?

@MikeMcMahon that happened to me too. Put Python before Perl in your path, so typing pip gets you the Python package manager.

I didn't have to do any of this, I just followed the instructions on pip-installer.org/en/latest/installing.html (basically you just write "python ez_setup.py" and then "python get-pip.py")

This is probably a dumb question. I entered python get-pip.py on a windows prompt but get "python is not recognized as an internal or external command, operable program or batch file." Am I doing something wrong? I have Python 2.7.8 installed. Do I need to change the directory? Thanks.

To start the "build in" pip you have to use python -m pip on windows!

python - How do I install pip on Windows? - Stack Overflow

python windows installation pip easy-install
Rectangle 27 1461

Python 2 2.7.8 and Python 3 3.3

Good news! Python 3.4 (released March 2014) and Python 2.7.9 (released December 2014) ship with Pip. This is the best feature of any Python release. It makes the community's wealth of libraries accessible to everyone. Newbies are no longer excluded from using community libraries by the prohibitive difficulty of setup. In shipping with a package manager, Python joins Ruby, Node.js, Haskell, Perl, Go--almost every other contemporary language with a majority open-source community. Thank you Python.

And, alas for everyone using Python 2.7.8 or earlier (a sizable portion of the community). There's no plan to ship Pip to you. Manual instructions follow.

Flying in the face of its 'batteries included' motto, Python ships without a package manager. To make matters worse, Pip was--until recently--ironically difficult to install.

Download get-pip.py, being careful to save it as a .py file rather than .txt. Then, run it from the command prompt:

python get-pip.py

You possibly need an administrator command prompt to do this. Follow Start a Command Prompt as an Administrator (Microsoft TechNet).

This installs the pip package, which (in Windows) contains ...\Scripts\pip.exe that path must be in PATH environment variable to use pip from the command line (see second part of 'Alternative Instructions' for adding it to your PATH,

The official documentation tells users to install Pip and each of its dependencies from source. That's tedious for the experienced, and prohibitively difficult for newbies.

For our sake, Christoph Gohlke prepares Windows installers (.msi) for popular Python packages. He builds installers for all Python versions, both 32 and 64 bit. You need to:

For me, this installed Pip at C:\Python27\Scripts\pip.exe. Find pip.exe on your computer, then add its folder (for example, C:\Python27\Scripts) to your path (Start / Edit environment variables). Now you should be able to run pip from the command line. Try installing a package:

pip install httpie

There you go (hopefully)! Solutions for common problems are given below:

If you work in an office, you might be behind a HTTP proxy. If so, set the environment variables http_proxy and https_proxy. Most Python applications (and other free software) respect these. Example syntax:

http://proxy_url:port
http://username:password@proxy_url:port

If you're really unlucky, your proxy might be a Microsoft NTLM proxy. Free software can't cope. The only solution is to install a free software friendly proxy that forwards to the nasty proxy. http://cntlm.sourceforge.net/

Python modules can be part written in C or C++. Pip tries to compile from source. If you don't have a C/C++ compiler installed and configured, you'll see this cryptic error message.

You can fix that by installing a C++ compiler such as MinGW or Visual C++. Microsoft actually ship one specifically for use with Python. Or try Microsoft Visual C++ Compiler for Python 2.7.

Is that "can be fixed and will not be fixed" or "cannot be fixed and will not be fixed"?

@MikeMcMahon that happened to me too. Put Python before Perl in your path, so typing pip gets you the Python package manager.

I didn't have to do any of this, I just followed the instructions on pip-installer.org/en/latest/installing.html (basically you just write "python ez_setup.py" and then "python get-pip.py")

This is probably a dumb question. I entered python get-pip.py on a windows prompt but get "python is not recognized as an internal or external command, operable program or batch file." Am I doing something wrong? I have Python 2.7.8 installed. Do I need to change the directory? Thanks.

To start the "build in" pip you have to use python -m pip on windows!

python - How do I install pip on Windows? - Stack Overflow

python windows installation pip easy-install
Rectangle 27 1458

Python 2 2.7.8 and Python 3 3.3

Good news! Python 3.4 (released March 2014) and Python 2.7.9 (released December 2014) ship with Pip. This is the best feature of any Python release. It makes the community's wealth of libraries accessible to everyone. Newbies are no longer excluded from using community libraries by the prohibitive difficulty of setup. In shipping with a package manager, Python joins Ruby, Node.js, Haskell, Perl, Go--almost every other contemporary language with a majority open-source community. Thank you Python.

And, alas for everyone using Python 2.7.8 or earlier (a sizable portion of the community). There's no plan to ship Pip to you. Manual instructions follow.

Flying in the face of its 'batteries included' motto, Python ships without a package manager. To make matters worse, Pip was--until recently--ironically difficult to install.

Download get-pip.py, being careful to save it as a .py file rather than .txt. Then, run it from the command prompt:

python get-pip.py

You possibly need an administrator command prompt to do this. Follow Start a Command Prompt as an Administrator (Microsoft TechNet).

This installs the pip package, which (in Windows) contains ...\Scripts\pip.exe that path must be in PATH environment variable to use pip from the command line (see second part of 'Alternative Instructions' for adding it to your PATH,

The official documentation tells users to install Pip and each of its dependencies from source. That's tedious for the experienced, and prohibitively difficult for newbies.

For our sake, Christoph Gohlke prepares Windows installers (.msi) for popular Python packages. He builds installers for all Python versions, both 32 and 64 bit. You need to:

For me, this installed Pip at C:\Python27\Scripts\pip.exe. Find pip.exe on your computer, then add its folder (for example, C:\Python27\Scripts) to your path (Start / Edit environment variables). Now you should be able to run pip from the command line. Try installing a package:

pip install httpie

There you go (hopefully)! Solutions for common problems are given below:

If you work in an office, you might be behind a HTTP proxy. If so, set the environment variables http_proxy and https_proxy. Most Python applications (and other free software) respect these. Example syntax:

http://proxy_url:port
http://username:password@proxy_url:port

If you're really unlucky, your proxy might be a Microsoft NTLM proxy. Free software can't cope. The only solution is to install a free software friendly proxy that forwards to the nasty proxy. http://cntlm.sourceforge.net/

Python modules can be part written in C or C++. Pip tries to compile from source. If you don't have a C/C++ compiler installed and configured, you'll see this cryptic error message.

You can fix that by installing a C++ compiler such as MinGW or Visual C++. Microsoft actually ship one specifically for use with Python. Or try Microsoft Visual C++ Compiler for Python 2.7.

Is that "can be fixed and will not be fixed" or "cannot be fixed and will not be fixed"?

@MikeMcMahon that happened to me too. Put Python before Perl in your path, so typing pip gets you the Python package manager.

I didn't have to do any of this, I just followed the instructions on pip-installer.org/en/latest/installing.html (basically you just write "python ez_setup.py" and then "python get-pip.py")

This is probably a dumb question. I entered python get-pip.py on a windows prompt but get "python is not recognized as an internal or external command, operable program or batch file." Am I doing something wrong? I have Python 2.7.8 installed. Do I need to change the directory? Thanks.

To start the "build in" pip you have to use python -m pip on windows!

python - How do I install pip on Windows? - Stack Overflow

python windows installation pip easy-install
Rectangle 27 11

PhantomJS doesn't currently support file downloads. Relevant issues with workarounds:

As far as I understand, you have at least 3 options:

casperjs
  • try with headless on xvfb

Here are also some links that might help too:

It seems file download capability has since been added to PhantomJS (including comments to that effect in the thread you linked to). However, I am not sure this is exposed in an easy-to-use way in Selenium.

Using Selenium with Python and PhantomJS to download file to filesyste...

python selenium selenium-webdriver download phantomjs
Rectangle 27 50

Copy and Paste the cv2.pyd file

Currently, I am using Windows 8.1 and 64-bit machine, Anaconda as IDE for Python 2.x.

Note: if you are on Windows 10 (or above) and if the below instruction works, please could you kindly add a comment? This will help out the community a lot! :-)

Anaconda is essentially a nicely packaged Python IDE that is shipped with tons of useful packages, such as NumPy, Pandas, IPython Notebook, etc. It seems to be recommended everywhere in the scientific community. Check out Anaconda to get it installed.

Cautious Note: I originally tried out installing the binstar.org opencv package, as suggested. That method however does not include the FFMPEG codec - i.e. you may be able to use OpenCV but you won't be able to process videos.

The following instruction works for me is inspired by this OpenCV Youtub video. So far I have got it working on both my Desktop and Laptop. Both 64-bit machines and Windows 8.1.

Firstly, go to the official OpenCV site to download the complete OpenCV package. Pick a version you like (2.x or 3.x). I am on Python 2.x and OpenCV 3.x - mainly because this is how the OpenCV-Python Tutorials are setup/based on.

C:\opencv

The Anaconda Site-packages directory (e.g. C:\Users\Johnny\Anaconda\Lib\site-packages in my case) contains the Python packages that you may import. Our goal is to copy and paste the cv2.pyd file to this directory (so that we can use the import cv2 in our Python codes.).

To do this, copy the cv2.pyd file...

# Python 2.7 and 32-bit machine: 
C:\opencv\build\python\2.7\x84

# Python 2.7 and 64-bit machine: 
C:\opencv\build\python\2.7\x64
C:\Users\Johnny\Anaconda\Lib\site-packages

After performing this step we shall now be able to use import cv2 in Python code. BUT, we still need to do a little bit more work to get FFMPEG (video codec) to work (to enable us to do things like processing videos.)

Add a new User Variable to point to the OpenCV (either x86 for 32-bit system or x64 for 64-bit system.) I am currently on a 64-bit machine.

| 32-bit or 64 bit machine? | Variable     | Value                                |
|---------------------------|--------------|--------------------------------------|
| 32-bit                    | `OPENCV_DIR` | `C:\opencv\build\x86\vc12`           |
| 64-bit                    | `OPENCV_DIR` | `C:\opencv\build\x64\vc12`           |
%OPENCV_DIR%\bin
PATH
C:\Users\Johnny\Anaconda;C:\Users\Johnny\Anaconda\Scripts;%OPENCV_DIR%\bin

We need to test whether we can now do these in Anaconda (via Spyder IDE):

To confrim that Anaconda is now able to import the OpenCV-Python package (namely, cv2), issue these in the IPython Console:

import cv2
print cv2.__version__

If the package cv2 is imported ok with no errors, and the cv2 version is printed out, then we are all good! Here is a snapshot:

Place a sample input_video.mp4 video file in a directory. We want to test whether we can:

.mp4
  • write out a new video file (can be .avi or .mp4 etc.)

To do this we need to have a test python code, call it test.py. Place it in the same directory as the sample input_video.mp4 file.

This is what test.py may look like (I've listed out both newer and older version codes here - do let us know which one works / not work for you!):

import cv2
cap = cv2.VideoCapture("input_video.mp4")
print cap.isOpened()   # True = read video successfully. False - fail to read video.

fourcc = cv2.VideoWriter_fourcc(*'XVID')
out = cv2.VideoWriter("output_video.avi", fourcc, 20.0, (640, 360))
print out.isOpened()  # True = write out video successfully. False - fail to write out video.

cap.release()
out.release()
import cv2
cv2.VideoCapture("input_video.mp4")
print cv2.isOpened()   # True = read video successfully. False - fail to read video.

fourcc = cv2.cv.CV_FOURCC(*'XVID')
out = cv2.VideoWriter("output_video.avi",fourcc, 20.0, (640,360))
print out.isOpened()  # True = write out video successfully. False - fail to write out video.

cap.release()
out.release()

This test is VERY IMPORTANT. If you'd like to process video files, you'd need to ensure that Anaconda / Spyder IDE can use the FFMPEG (video codec). It took me days to have got it working. But I hope it would take you much less time! :)

Note: one more very important tip when using the Anaconda Spyder IDE. Make sure you check the Current Working Directory (CWD)!!!

To use OpenCV fully with Anaconda (and Spyder IDE), we need to:

  • Copy and paste the cv2.pyd to the Anaconda site-packages directory.
  • Set user environmental variables so that Anaconda knows where to find the FFMPEG utility.

Brilliant answer! Note that if you're using conda environments, cv2.pyd should be added to the environment's site-packages folder (e.g. C:\Users\cod3monk3y\Anaconda\envs\foo\Lib\site-packages\cv2.pyd). Also worth noting, the .pyd file is just a Windows DLL with a specific interface to play nicely with Python.

the code you posted above prints out true, true for me but the output is an empty 6kb video file. However, the code below writes properly to a file. (how do i get line breaks in these comments?)

If i could up it more i would, best answer ever. I would just add, if you've installed python, uninstall it, conda has its own

Thanks for this! One more suggested test that took me awhile to figure out. After confirming that the video can be opened, I wanted to confirm that I could extract a frame (frame 100) as an array. In openCV 2.4.9, the command for this was cap.set(cv2.cv.CV_CAP_PROP_POS_FRAMES, 100). In openCV 3.2.0, the equivalent command has changed to cap.set(cv2.CAP_PROP_POS_FRAMES, 100)

Thank you sir for this awesome and well explained answer. You are making the life easier for many newbies sir !

How to install python opencv through Conda? - Stack Overflow

python opencv anaconda conda
Rectangle 27 2

I don't think he wants a programming solution

On second thoughts, if you're saving the files, urlretrieve in urllib is probably easier.

@doukremt: he asked a question specifically about Python on a programming Q&A site. Why would you assume he doesn't want to program?

web - Download Files from a Website with Python - Stack Overflow

python web download
Rectangle 27 16

The best method for installing FeedParser (or almost any Python package) is by using pip, Python's package manager. pip will be installed by default alongside Python as of Python 3.4 (and Python 2.7.9). Since the latest version of Python (as of time of editing) is Python 3.5, this is really the approach you should be taking whenever possible.

To run pip, do the following:

  • Open the command line. You can do this by clicking start, then typing cmd in the run textbox.
  • Make sure you are connected to the internet.
  • Type in pip install feedparser and hit enter (and wait for pip to download and install FeedParser)

You're done! Open up IDLE and type in import feedparser to confirm.

pip
C:\Python34\Scripts\pip install feedparser
  • Closing the command line, and re-opening it in admin mode (click start, type 'cmd', right-click, select "Run as Administrator"), if it's failing due to permission errors

If you would like to learn more about pip, or need help setting it up on older versions of Python, see this post.

If, for some reason, you are unable to use pip, you can always try doing a manual install.

Feedburner needs a module named setuptools in order to install. Unfortunately, setuptools isn't compatible with Python 3.x, but you can use something called Distribute as a convenient replacement. If you already have this installed, you can skip Part A.

Navigate in the command line to where the extracted feedparser folder is by using the cd command. For example, on my computer, I typed:

cd C:\Users\Michael0x2a\Downloads\feedparser-5.1\feedparser-5.1

(Your version number will probably differ. For example, you might have feedparser-5.3 or something)

Make sure the folder you're in has the readme, various other docs, and setup.py. To check, type dir into the command line. It should list the files and folders there.

python setup.py install
C:\Python34\python.exe setup.py install
import feedparser

THANK YOU THANK YOU THANK YOU SO FREAKING MUCH!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! You gave me exactly what I needed!!!!!!!!!!!!! I found examples similar but they all skipped steps and I became so confused you are the best Thank You So Much!

How to install Feedparser 5.1 for Python 3.2 Windows 64 bit. - Stack O...

python install feedparser windows64
Rectangle 27 2

You would have to use another language. You should see Save file Javascript with file name (this requires you creating it on your website and having the user download it) and Python Save to file (for implementing Python in HTML see here). Or in Javascript you could use ActiveX to save a file by doing*:

function WriteToFile()
{
var fso = new ActiveXObject("Scripting.FileSystemObject");
var s = fso.CreateTextFile("C:\\Test.txt", true);
s.WriteLine('Hello');
s.Close();
}
myFile = open('Failed.py', 'w')
myFile.write('whatever')
myFile.close()
uriContent = "data:application/octet-stream;filename=filename.txt," + 
          encodeURIComponent(codeMirror.getValue());
newWindow=window.open(uriContent, 'filename.txt');
ActiveX

how would I invoke the python script? with javascript I just put the code between two "<script>" tags.

with open('Failed.py','w') as myFile: myFile.write('whatever')

@Yevgen Thanks for the tip :), I don't actually do python, but since this question required more javascript/html knowledge, I thought I'd answer it anyways

javascript - How to create a file and append data from html page ? - S...

javascript python html file server-side
Rectangle 27 10

I'd like to offer an updated Python 3 version of Vishal's excellent answer, which was using Python 2, along with some explanation of the adaptations / changes, which may have been already mentioned.

from io import BytesIO
from zipfile import ZipFile
import urllib.request

    url = urllib.request.urlopen("http://www.unece.org/fileadmin/DAM/cefact/locode/loc162txt.zip")

    with ZipFile(BytesIO(url.read())) as my_zip_file:
        for contained_file in my_zip_file.namelist():
            # with open(("unzipped_and_read_" + contained_file + ".file"), "wb") as output:
            for line in my_zip_file.open(contained_file).readlines():
                print(line)
                # output.write(line)
  • There's no StringIO in Python 3. Instead, I use io, and from it I import BytesIO, because we will be handling a bytestream -- Docs, also this thread.
  • "The legacy urllib.urlopen function from Python 2.6 and earlier has been discontinued; urllib.request.urlopen() corresponds to the old urllib2.urlopen.", Docs.
  • In Python 3, the printed output lines will look like so: b'some text'. This is expected, as they aren't strings - remember, we're reading a bytestream. Have a look at Dan04's excellent answer.
with ... as
zipfile = ...
  • The script now uses namelist() to cycle through all the files in the zip and print their contents.
  • I moved the creation of the ZipFile object into the with-statement, altough I'm not sure if that's better.
  • I added (and commented out) an option to write the bytestream to file (per file in the zip), in response to NumenorForLife's comment; it adds "unzipped_and_read_" to the beginning of the filename and a ".file" extension (I prefer not to use ".txt" for files with bytestrings). The indenting of the code will, of course, need to be adjusted if you want to use it. Need to be careful here -- because we have a byte string, we use binary mode, so "wb"; I have a feeling that writing binary opens a can of worms anyway...
  • NumenorForLife asked about saving the zip to disk. I'm not sure what he meant by it -- downloading the zip file? That's a different task; see Oleh Prypin's excellent answer.

Here's a way:

import urllib.request
import shutil

with urllib.request.urlopen("http://www.unece.org/fileadmin/DAM/cefact/locode/2015-2_UNLOCODE_SecretariatNotes.pdf") as response, open("downloaded_file.pdf", 'w') as out_file:
    shutil.copyfileobj(response, out_file)

unzip - Python: downloading and unzipping a .zip file without writing ...

python unzip
Rectangle 27 10

I'd like to offer an updated Python 3 version of Vishal's excellent answer, which was using Python 2, along with some explanation of the adaptations / changes, which may have been already mentioned.

from io import BytesIO
from zipfile import ZipFile
import urllib.request

    url = urllib.request.urlopen("http://www.unece.org/fileadmin/DAM/cefact/locode/loc162txt.zip")

    with ZipFile(BytesIO(url.read())) as my_zip_file:
        for contained_file in my_zip_file.namelist():
            # with open(("unzipped_and_read_" + contained_file + ".file"), "wb") as output:
            for line in my_zip_file.open(contained_file).readlines():
                print(line)
                # output.write(line)
  • There's no StringIO in Python 3. Instead, I use io, and from it I import BytesIO, because we will be handling a bytestream -- Docs, also this thread.
  • "The legacy urllib.urlopen function from Python 2.6 and earlier has been discontinued; urllib.request.urlopen() corresponds to the old urllib2.urlopen.", Docs.
  • In Python 3, the printed output lines will look like so: b'some text'. This is expected, as they aren't strings - remember, we're reading a bytestream. Have a look at Dan04's excellent answer.
with ... as
zipfile = ...
  • The script now uses namelist() to cycle through all the files in the zip and print their contents.
  • I moved the creation of the ZipFile object into the with-statement, altough I'm not sure if that's better.
  • I added (and commented out) an option to write the bytestream to file (per file in the zip), in response to NumenorForLife's comment; it adds "unzipped_and_read_" to the beginning of the filename and a ".file" extension (I prefer not to use ".txt" for files with bytestrings). The indenting of the code will, of course, need to be adjusted if you want to use it. Need to be careful here -- because we have a byte string, we use binary mode, so "wb"; I have a feeling that writing binary opens a can of worms anyway...
  • NumenorForLife asked about saving the zip to disk. I'm not sure what he meant by it -- downloading the zip file? That's a different task; see Oleh Prypin's excellent answer.

Here's a way:

import urllib.request
import shutil

with urllib.request.urlopen("http://www.unece.org/fileadmin/DAM/cefact/locode/2015-2_UNLOCODE_SecretariatNotes.pdf") as response, open("downloaded_file.pdf", 'w') as out_file:
    shutil.copyfileobj(response, out_file)

unzip - Python: downloading and unzipping a .zip file without writing ...

python unzip
Rectangle 27 5

When you download a package with setup.py open your Terminal (Mac,Linux) or Command Prompt (Windows). Using cd and helping you with Tab button set the path right to the folder where you have downloaded the file and where there is setup.py :

iMac:~ user $ cd path/pakagefolderwithsetupfile/
python setup.py install
iMac:pakagefolderwithsetupfile user$ python setup.py install
enter

python - What is setup.py? - Stack Overflow

python setup.py pypi python-packaging
Rectangle 27 5

When you download a package with setup.py open your Terminal (Mac,Linux) or Command Prompt (Windows). Using cd and helping you with Tab button set the path right to the folder where you have downloaded the file and where there is setup.py :

iMac:~ user $ cd path/pakagefolderwithsetupfile/
python setup.py install
iMac:pakagefolderwithsetupfile user$ python setup.py install
enter

python - What is setup.py? - Stack Overflow

python python-3.x setup.py pypi python-packaging
Rectangle 27 8

Rather than going the harder route of downloading specific packages. I prefer to go the faster route of using Conda. pip has its issues.

C:\Users\xyz>conda install -c anaconda scipy=0.18.1
Fetching package metadata .............
Solving package specifications:

Package plan for installation in environment C:\Users\xyz\Miniconda3:

mkl:       2017.0.1-0         anaconda
numpy:     1.12.0-py36_0      anaconda
scipy:     0.18.1-np112py36_1 anaconda

The following packages will be SUPERCEDED by a higher-priority channel:

conda:     4.3.11-py36_0               --> 4.3.11-py36_0 anaconda
conda-env: 2.6.0-0                     --> 2.6.0-0       anaconda
conda-env-2.6. 100% |###############################| Time: 0:00:00  32.92 kB/s
mkl-2017.0.1-0 100% |###############################| Time: 0:00:24   5.45 MB/s
numpy-1.12.0-p 100% |###############################| Time: 0:00:00   5.09 MB/s
scipy-0.18.1-n 100% |###############################| Time: 0:00:02   5.59 MB/s
conda-4.3.11-p 100% |###############################| Time: 0:00:00   4.70 MB/s

python - Can't install Scipy through pip - Stack Overflow

python scipy
Rectangle 27 8

Rather than going the harder route of downloading specific packages. I prefer to go the faster route of using Conda. pip has its issues.

C:\Users\xyz>conda install -c anaconda scipy=0.18.1
Fetching package metadata .............
Solving package specifications:

Package plan for installation in environment C:\Users\xyz\Miniconda3:

mkl:       2017.0.1-0         anaconda
numpy:     1.12.0-py36_0      anaconda
scipy:     0.18.1-np112py36_1 anaconda

The following packages will be SUPERCEDED by a higher-priority channel:

conda:     4.3.11-py36_0               --> 4.3.11-py36_0 anaconda
conda-env: 2.6.0-0                     --> 2.6.0-0       anaconda
conda-env-2.6. 100% |###############################| Time: 0:00:00  32.92 kB/s
mkl-2017.0.1-0 100% |###############################| Time: 0:00:24   5.45 MB/s
numpy-1.12.0-p 100% |###############################| Time: 0:00:00   5.09 MB/s
scipy-0.18.1-n 100% |###############################| Time: 0:00:02   5.59 MB/s
conda-4.3.11-p 100% |###############################| Time: 0:00:00   4.70 MB/s

python - Can't install Scipy through pip - Stack Overflow

python scipy
Rectangle 27 2

Your code is functional. The problem is how you are running it. You mentioned in your comments that you are using PythonAnywhere. PythonAnywhere is not set up to do anything graphical. It will download the image into the correct directory, but PIL will not function correctly with PythonAnywhere.

import urllib

if __name__ == "__main__":
    droste = urllib.urlopen("http://is.gd/cHqT")
    with open("droste.png", "wb") as imgFile:
        imgFile.write(droste.read())
    print "Got it!"

    print "Now lets test if it really exists..."
    try:
        with open("droste.png", "rb") as imgFile:
            pass
        print "There were no errors so the file exists"
    except:
        print "ERROR: image was not saved properly!"

If you start up a BASH session with PythonAnywhere, you will see that the file droste.png exists, and you can download it to your computer and view it. Your program is OK.

If you really want to use your program, or get serious about python programming. You really should install Python locally to your computer. If you want to keep your code in the cloud, then use dropbox, github, or bitbucket. PythonAnywhere has uses, but normally you will just want to have python on your computer.

file io - Open Image Error Python - Stack Overflow

python file-io io ioerror
Rectangle 27 13

We also develop software projects that depend on numpy, scipy and other PyPI packages. Hands down, the best tool currently available out there for managing remote installations is zc.buildout. It is very easy to use. You download a bootstrapping script from their website and distribute that with your package. You write a "local deployment" file, called normally buildout.cfg, that explains how to install the package locally. You ship both the bootstrap.py file and buildout.cfg with your package - we use the MANIFEST.in file in our python packages to force the embedding of these two files with the zip or tar balls distributed by PyPI. When the user unpackages it, it should execute two commands:

$ python bootstrap.py # this will download zc.buildout and setuptools
$ ./bin/buildout # this will build and **locally** install your package + deps

The package is compiled and all dependencies are installed locally, which means that the user installing your package doesn't even need root privileges, which is an added feature. The scripts are (normally) placed under ./bin, so the user can just execute them after that. zc.buildout uses setuptools for interaction with PyPI so everything you expect works out of the box.

You can extend zc.buildout quite easily if all that power is not enough - you create the so-called "recipes" that can help the user to create extra configuration files, download other stuff from the net or instantiate custom programs. zc.buildout website contains a video tutorial that explains in details how to use buildout and how to extend it. Our project Bob makes extensive use of buildout for distributing packages for scientific usage. If you would like, please visit the following page that contains detailed instructions for our developers on how they can setup their python packages so other people can build and install them locally using zc.buildout.

Great answer, I'll definitely look into buildout as well. Although our Python tool doesn't depend on other Python packages yet (exactly to avoid installation problems), it will in the near future, so we'll need a decent solution to allow users to easily install our tool. zc.buildout looks like a great solution, but it does still require the user to 1) download the package, 2) unpack it, 3) run 'python bootstrap.py' and 4) run buildout. Could you somehow make it work with one command that does it all, e.g., write a nastly setup.py script to make 'easy_install foo' work with buildout)?

I don't think it is easily possible nor that you would like that as something you'd have to maintain. Buildout depends on setuptools and not the contrary. In my experience, the best is always to rely on PyPI for package distribution: it is there, costs nothing and it is mirrored. We put a link to the PyPI package page on your publications. The user hits it and sees a nice manual and a green download button. You can write a script that does all this in a single shot, but then you will need to distribute/maintain that separately.

In our case, we already have a meta package that combines the three packages that form our software, see pypi.python.org/pypi/easybuild. The all-in-one-go script could be a part of the meta package (which now only has a setup.py basically).

The user still needs to download easybuild, unpack it and then run something. Why can't this something be buildout?

It could, but I think a solution that requires only running a single command would work as well.

pip - How to easily distribute Python software that has Python module ...

python pip setuptools package-managers distribute
Rectangle 27 2

Make sure that you are serving protovis/protovis.js using your BaseHTTPServer. Basically the browser will "see" the line <script type="text/javascript" src="protovis/protovis.js"></script> when it is reading the html output it gets, and then request that the server actually send the protovis/protovis.js file. I didn't see the python code in that github download, so I can't specifically show how to do this in the context of your code, but check out SimpleHTTPRequestHandler. Using that, you can add to the do_GET() method to have the server send protovis/protovis.js when it is requested (that is, when self.path is protovis/protovis.js).

Yes, I've followed your instruction. I've added a method into do_GET() which reads protovis/protovis.js It works now. Thank you very much

python - baseHTTPserver cannot use javascript library - Stack Overflow

javascript python protovis basehttpserver
Rectangle 27 31

The reason it's not working is because these python packages are only wrappers to tesseract. You have to compile tesseract using a AWS Linux instance and copy the binaries and libraries to the zip file of the lambda function.

1) Start an EC2 instance with 64-bit Amazon Linux;

sudo yum install gcc gcc-c++ make
sudo yum install autoconf aclocal automake
sudo yum install libtool
sudo yum install libjpeg-devel libpng-devel libpng-devel libtiff-devel zlib-devel

3) Compile and install leptonica:

cd ~
mkdir leptonica
cd leptonica
wget http://www.leptonica.com/source/leptonica-1.73.tar.gz
tar -zxvf leptonica-1.73.tar.gz
cd leptonica-1.73
./configure
make
sudo make install

4) Compile and install tesseract

cd ~
mkdir tesseract
cd tesseract
wget https://github.com/tesseract-ocr/tesseract/archive/3.04.01.tar.gz
tar -zxvf 3.04.01.tar.gz
cd tesseract-3.04.01
./autogen.sh
./configure
make
sudo make install
cd /usr/local/share/tessdata
wget https://github.com/tesseract-ocr/tessdata/raw/3.04.00/eng.traineddata
export TESSDATA_PREFIX=/usr/local/share/

At this point you should be able to use tesseract on this EC2 instance. To copy the binaries of tesseract and use it on a lambda function you will need to copy some files from this instance to the zip file you upload to lambda. I'll post all the commands to get a zip file with all the files you need.

6) Zip all the stuff you need to run tesseract on lambda

cd ~
mkdir tesseract-lambda
cd tesseract-lambda
cp /usr/local/bin/tesseract .
mkdir lib
cd lib
cp /usr/local/lib/libtesseract.so.3 .
cp /usr/local/lib/liblept.so.5 .
cp /usr/lib64/libpng12.so.0 .
cd ..

mkdir tessdata
cd tessdata
cp /usr/local/share/tessdata/eng.traineddata .
cd ..

cd ..
zip -r tesseract-lambda.zip tesseract-lambda

The tesseract-lambda.zip file have everything lambda needs to run tesseract. The last thing to do is add the lambda function at the root of the zip file and upload it to lambda. Here is an example that I have not tested, but should work.

7) Write a lambda function like this one and add it on the root of tesseract-lambda.zip:

from __future__ import print_function

import urllib
import boto3
import os
import subprocess

SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
LIB_DIR = os.path.join(SCRIPT_DIR, 'lib')

s3 = boto3.client('s3')

def lambda_handler(event, context):

    # Get the bucket and object from the event
    bucket = event['Records'][0]['s3']['bucket']['name']
    key = urllib.unquote_plus(event['Records'][0]['s3']['object']['key']).decode('utf8')

    try:
        print("Bucket: " + bucket)
        print("Key: " + key)

        imgfilepath = '/tmp/image.png'
        jsonfilepath = '/tmp/result.txt'
        exportfile = key + '.txt'

        print("Export: " + exportfile)

        s3.download_file(bucket, key, imgfilepath)

        command = 'LD_LIBRARY_PATH={} TESSDATA_PREFIX={} {}/tesseract {} {}'.format(
            LIB_DIR,
            SCRIPT_DIR,
            SCRIPT_DIR,
            imgfilepath,
            jsonfilepath,
        )

        try:
            output = subprocess.check_output(command, shell=True)
            print(output)
            s3.upload_file(jsonfilepath, bucket, exportfile)
        except subprocess.CalledProcessError as e:
            print(e.output)

    except Exception as e:
        print(e)
        print('Error processing object {} from bucket {}.'.format(key, bucket))
        raise e

From time to time things change in AWS Lambda's environment. For example, the current image for the lambda env is amzn-ami-hvm-2017.03.1.20170812-x86_64-gp2 (it might not be this one when you read this answer). If tesseract starts to return segmentation fault, run "ldd tesseract" on the Lambda function and see the output for what libs are needed (currently libtesseract.so.3 liblept.so.5 libpng12.so.0).

Is something like this what I would need to do to get the ODBC driver to be found when trying to use pypyodbc to connect to an RDS instance of SQL Server? pypyODBC raises an error when it can't find an ODBC library.(driver?)

Small correction to Jose's answer: the tool to get a list of shared dependencies is ldd

In the mean time the training data in the master branch ist for tesseract 4.0, which doesn't work with 3.04. For 3.04 you need to get the training data from github.com/tesseract-ocr/tessdata/raw/3.04.00/eng.traineddata

errorMessage": "Command 'LD_LIBRARY_PATH=/var/task/lib TESSDATA_PREFIX=/var/task /var/task/tesseract -v' returned non-zero exit status -11
tesseract -v

I wanted to use this tool with the lastest lambda (amzn-ami-hvm-2017.03.1.20170812-x86_64-gp2) because it returned Segmentation fault, so I had to fix it: (1) clone repo (2) $ LD_LIBRARY_PATH=./ ldd tesseract to find NotFound libraries (3) ONLY these libraries are required: liblept.so.5, libpng12.so.0 and libtesseract.so.3 (4) It worked again! No need to copy any other one!

python - Tesseract OCR on AWS Lambda via virtualenv - Stack Overflow

python amazon-web-services virtualenv tesseract aws-lambda
Rectangle 27 2

lxml uses libxml2, libxslt (in background) but libxml2, libxslt are not Python modules - it's C/C++ libraries. So you can't install them using pip. You have to download and install them manually.

lxml
  • cp35 in file name means version for Python 3.5.
  • win32 in file name means version for 32bit Python.
  • amd64 in file name means version for 64bit Python.

You probably use 32bit Python because I see 32 and (x86) in your path

C:\program files (x86)\python35-32\python.exe

You should find links to libxml2, libxslt on this page too. This libraries has .dll and .exe files and you can put them in any folder which is in PATH variable. Libraries mostly are installed in C:\Windows or in subfolder.

libxml2, libxslt may have also C/C++ header files *.h which you may need when you compile lxml. (on Linux this files are in separated packages libxml2-dev, libxslt-dev)

BTW: You can use Anaconda distribution (instead of Python.org distribution). It installs the same Python but with some precompiled modules - ie. it installs lxml automatically. Anaconda can be the best solution for Windows users.

Installing lxml, libxml2, libxslt for Python 3.5 on Windows 10 - Stack...

python lxml
Rectangle 27 4

If, like me, you want to make a function pack or something that people can download then it's very simple. Just write your function in a python file and save it as the name you want IN YOUR PYTHON DIRECTORY. Now, in your script where you want to use this, you type:

from FILE NAME import FUNCTION NAME

Note - the parts in capital letters are where you type the file name and function name.

def choose(a):
  from random import randint
  b = randint(0, len(a) - 1)
  c = a[b]
  return(c)
from function_choose import choose
list_a = ["dog", "cat", "chicken"]
print(choose(list_a))

python - Importing a function from a class in another file? - Stack Ov...

python class import