Python Packages and Indigo

This page describes how Indigo 2022.2.0 works with Python, specifically with respect to packages/libraries.

Indigo 2022.2 shipped with API version 3.2 and is the first version to contain support for Python version 3 (check the API Version Chart to see what version of the API is available in which Indigo versions). With this version of the API, Python 2 and Python 3 are now automatically installed as part of the Indigo installer. For Indigo 2022.2, we are including both Python 3.10.2 and Python 2.7.18 (for backwards compatibility - it is an x86 install and will require Rosetta to run on Apple Silicon Macs). The Python 2.7.18 install was specifically added because macOS 12.3 removed its Python install and is a transitional step only. Python 2 and support for APIs < 3.0 will be removed in Indigo 2022.2 later this year.

You may know that Indigo plugins run in a process called the IndigoPluginHost (IPH). With this release, there are now 2 versions of the IPH process: IndigoPluginHost (IPH2 for clarity), which runs the legacy Python 2 API, and IndigoPluginHost3 (IPH3) which runs the new Python 3 based API. We determine which version to use when starting a plugin (by which API the plugin specifies). API 3.0 and later will run the IPH3 process, anything earlier will run the older IPH2 process.

For this release, all embedded and external Python scripts will be run in IPH3 (and therefore Python 3) - so you will need to update your scripts if they aren't working (we can't automatically tell if they will work or not unfortunately). We believe that simple scripts will most likely work without change, but if not, you can walk through the document mentioned above and perhaps spot the changes you need. You can also post your scripts on the Help Converting to Python 3 forum to get help with any conversion issues.

The Indigo 2022.2 installer includes the Python 2.7.18 installer (which is Intel only) and the Python 3.10.2 installer (includes Intel and Apple Silicon support) from https://www.python.org - the official Mac installers for the Python language. As a result, any installs of other versions of Python from that website may cause unexpected results in Indigo and potentially cause it to break. We've taken all precautions we can to try and avoid conflicts, but there is only so much we can do. So please use caution when installing other Python versions to your server machine. If you have any questions about other Python versions, post your questions on our forums.

Installing Additional Python Packages with Indigo 2022.2

As noted above, Indigo 2022.2 ships with two versions of Python. Each version contains several modules in addition to Python's standard core modules. The specific modules installed with each version of Python are listed below. However, in some instances, you may need to manually install one or more additional Python packages to support a third-party plugin or your own custom scripts. The manner in which this is done, depends on the Indigo API version that the plugin supports.

We were able to extend support for Python 2 as a convenience for our users. As a result, Indigo 2022.2 will continue to support Python 2 and Python 3. Users should plan for Python 2 support to be dropped in Indigo 2023.1.

For plugins that use Python 3, use pip3 install [package] commands to install packages:

Last login: Tue jan 1 12:34:56 on console
user@my mac ~ % pip3 install tensorflow
...

For plugins that use Python 2, use pip2 install [package] commands to install packages:

Last login: Tue jan 1 12:34:56 on console
user@my mac ~ % pip2 install pytorch
...

To see which packages are installed, use the list command:

Last login: Tue jan 1 12:34:56 on ttys000
user@my mac ~ % pip3 list
Package            Version
------------------ ---------
astroid            2.9.3
certifi            2021.10.8
charset-normalizer 2.0.11
cycler             0.11.0
fonttools          4.29.1
...
user@my mac ~ %

Note: Some Python packages – and specifically those that are included in IPH3 as described below – will not appear in the pip3 list output.

To see more detailed information about an individual package, use the show command.

Last login: Sun Mar 27 10:20:04 on ttys000
user@my mac ~ % pip3 show requests
Name: requests
Version: 2.27.1
Summary: Python HTTP for Humans.
Home-page: https://requests.readthedocs.io
Author: Kenneth Reitz
Author-email: me@kennethreitz.org
License: Apache 2.0
Location: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages
Requires: charset-normalizer, certifi, idna, urllib3
Required-by: 
user@my mac ~ % 

Particularly helpful in the show output is the dependencies information – both what the package requires and what requires the package. Note that the dependencies information may not include information about packages that are included with the IPH.

Installing additional Python versions on the Indigo 2022.2 server machine is NOT recommended (see above). Installing additional Python modules should only be done when necessary.

Python Packages for Plugin Developers

Because the packages included with Indigo should not be modified (to ensure a smoothly running server), plugin developers must include any other desired version within the plugin package. Plugin developers should note that some Python packages will not integrate universally across all installations as they rely on code compiled for specific hardware configurations (for example, Intel vs. Apple silicon). The IPH processes use the following Module Search Paths (Python interpreters use PYTHONPATH to describe the order of directories to search for installed packages, the first matching package will be used):

The PYTHONPATH for IPH3 is:

  1. Contents/Server Plugin (inside the plugin bundle)
  2. Contents/Packages (inside the plugin bundle)
  3. IndigoPluginHost3 (inside the app itself)
  4. /Library/Application Support/Perceptive Automation/Python3-includes
  5. /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages

and the PYTHONPATH for IPH2 is:

  1. Contents/Server Plugin (inside the plugin bundle)
  2. IndigoPluginHost (inside the app itself)
  3. /Library/Application Support/Perceptive Automation/Python2-includes
  4. /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages

Packages are installed in three ways:

  1. Some packages are included within the IPH itself. These are “protected” and can not be modified (listed below).
  2. You may add modules/packages that you develop yourself to the Python#-includes folders above, and those will be shared across all Indigo Plugins and Scripts.
  3. Other packages are installed using a pip3 (or pip2) process. Warning: Indigo will preinstall several packages (listed below) at install time. You should avoid modifying those unless absolutely necessary as other 3rd party plugins may rely on them.

Access to IPH3 Libraries

There are some important implications that result from having some libraries included in the IPH3. Namely, those packages will be available to plugins, Indigo scripts (embedded and linked), the Indigo python shell (from the menu item or by doing indigo-host in a terminal window), but IPH3 libraries will NOT be available if you start the python interpreter directly (by doing python2 or python3 in a terminal window). Refer to this chart to determine accessibility:

Method Access to
IPH3 Libraries
plugins yes
embedded scripts yes
linked scripts yes
Indigo python shell using Indigo menu yes
Indigo python shell using Terminal indigo-host yes
Mac Terminal using python3 command no

Python Packages for Scripters

In this release, all Python scripts (either embedded or linked) are executed using the IPH3 process using Python 3. Therefore, the server will be able to locate any packages your scripts use in the same way as described above:

  1. IndigoPluginHost3
  2. /Library/Application Support/Perceptive Automation/Python3-includes
  3. /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages

If you need a module/package for your plugin, you can install it via pip3 as shown above, which will install it into the Python site-packages folder. We recommend (but don't require) the following folders be used for custom scripts and modules:

  • /Library/Application Support/Perceptive Automation/Scripts – any linked scripts you use for Indigo Schedules and Actions using the “Execute Script (Script and File Actions” command should be stored in this folder.
  • /Library/Application Support/Perceptive Automation/Python3-includes – any modules you have developed (a module is a file that contains classes, functions, constants, etc.) that you want to share across multiple scripts, should be stored in the /Library/Application Support/Perceptive Automation/Python3-includes folder. (Indigo versions prior to 2022 should use the /Library/Application Support/Perceptive Automation/Python2-includes folder).

Warning 1: as mentioned above, Indigo will preinstall several packages (listed below) at install time. You should avoid modifying those unless absolutely necessary as other 3rd party plugins may rely on them.

Warning 2: you should use caution when naming your custom scripts to ensure they don't supplant other Python packages. For example, if you install the holidays Python package using pip3 install holidays, it will be installed in the site-packages folder. If you also have a custom script in the Python3-includes folder named holidays.py, the IPH will import your script instead of the site-packages version (because the Python3-includes folder is searched before the site-packages folder).

If Something Goes Wrong

To provide users with a measure of safety regarding the Python packages that Indigo installs, the Indigo installer also includes the capacity to repair the Python installation if something should become damaged. For example, if a user were to accidentally upgrade one of the Python packages – and as a result cause the Indigo server to stop functioning properly – re-running the Indigo installer may be able to bring things back under control. There's a few things you can try should you run into conflicts between Python versions.

Re-running the Indigo installer on an existing server shouldn't affect user-installed libraries (existing user-installed libraries should be unaffected).

Moving Indigo to a New Machine

Depending on the method used, any Python modules installed via pip3 may need to be re-installed on the new machine. For more information, consult Moving an Indigo Installation to Another Mac.

Python 3.10.2 Packages Installed by Indigo

Indigo 2022.2 ships with support for the following Python 3 packages in addition to the packages installed with the standard Python framework. Generally, if earlier versions of the listed packages are installed at the location Indigo uses, they will be upgraded to the version numbers below.

Included inside IPH 3
Unmodifiable
PIP installed by Indigo
PIP Modifiable (but not recommended)
Python Package Version Python Package Version
py-applescript 1.0.3 cycler 0.11.0
certifi 021.10.08 cryptography 39.0.2
charset_normalizer 2.0.12 - cffi 1.15.1
python-dateutil 2.8.2 - pycparser 2.21
idna 3.3 dictdiffer 0.9.0
jedi 0.18.1 fonttools 4.31.2
oauthlib 3.2.0 future 0.18.2
packaging 21.3 httpx 0.23.3
parso 0.8.3 janus 1.0.0
pudb 2022.1.0 - typing_extensions 4.4.0
pyaes 1.3.0 Jinja2 3.1.2
pygments 2.11.2 kiwisolver 1.4.0
pyparsing 3.0.7 MarkupSafe 2.1.1
pyserial 3.5 matplotlib 3.5.1
requests 2.27.1 numpy 1.22.3
requests-oauthlib 1.3.1 packaging 21.3
urllib3 1.26.8 Pillow 9.0.1
urwid 2.1.2 pyparsing 3.0.7
xmljson 0.2.1 python-dateutil 2.8.2
pyobjc 8.4.1
sanic 22.6.1
- aiofiles 22.1.0
- httptools 0.5.0
- multidict 6.0.4
- sanic-routing 22.3.0
- sanic-session 0.8.0
- ujson 5.7.0
- uvloop 0.17.0
- websockets 10.4
scipy 1.8.0
six 1.16.0

Python 2.7.18 Packages Installed by Indigo

Indigo 2022.1 ships with the following Python 2 packages in addition to the packages installed with the standard Python framework.

Included inside IPH 2
Unmodifiable
PIP installed by Indigo
PIP Modifiable (but not recommended)
Python Package Version Python Package Version
enum 1.1.6 backports.functools-lru-cache 1.6.4
oauthlib 2.0.2 cycler 0.10.0
requests 2.10.0 future 0.18.2
requests-oauthlib 0.8.0 kiwisolver 1.1.0
pudb 2016.1 matplotlib 2.2.5
py-applescript 1.0.3 numpy 1.16.6
pyaes 1.3.0 pyparsing 2.4.7
pygments 2.1.3 python-dateutil 2.8.2
pyserial 3.1.1 pytz 2022.1
python-dateutil 2.8.2 scipy 1.2.3
six 1.16.0 six 1.16.0
urwid 1.3.1 subprocess32 3.5.4
xmljson 0.1.7
indigo_2022.2_documentation/python_packages.txt · Last modified: 2024/01/26 15:46 by davel17
 

© Perceptive Automation, LLC. · Privacy