Rust and Python

Oxidize Your Snake

Sven-Hendrik Haase

svenstaro.org

This talk

  • Motivation
  • Current approaches
  • Approach using Rust
  • Takeaways

Motivation

  • Speed up your program
  • Escape the GIL (multithreading)
  • Bind to existing system-libraries

Ways to speed up Python

Manual/C: Python C extensions, ctypes, cffi

Binding generator: Boost.Python, SWIG, PyO3

Specialized lib: pillow, numpy

JIT: pypy

Python-like dialect: cython

Problems of current approaches

Unsafe

C is not ergonomic

JIT unreliable

Multithreading

Instead, oxidize your snake

What is Rust?

  • Safe
  • Modern
  • Fast
  • Statically and strongly typed
  • Immutable by default
  • Private by default
  • Amazing tooling
  • Great community
  • Cute mascot
  • FEARLESS CONCURRENCY

What Rust looks like

Hello, Alice
Hello, Bob
Hello, Charlie
                            
Hello, Alice
Hello, Bob
Hello, Charlie

Rust and Python

We'll use PyO3

Write Python modules in Rust

Use Python from Rust

Python module in Rust

$ python europython.py
Hello, Alisa 💖!

Structs as Python classes 1/2

Structs as Python classes 2/2

$ python classes.py
4

Let's calculate π using Monte Carlo

Some slow Python code

Very crappy way to estimate π. 

$ time python slow.py
3.141624668
python slow.py  247.18s user 0.01s system 99% cpu 4:07.36 total

We need to go faster

Some faster Rust code 1/2

Some faster Rust code 2/2

$ time python with_rust.py
3.141672068
python with_rust.py  4.95s user 0.00s system 99% cpu 4.966 total

We need to go even faster

Multithreading in Rust 1/2

Multithreading in Rust 2/2

$ time python multithreading.py
3.142886888
python multithreading.py  13.65s user 0.01s system 783% cpu 1.742 total

How to go faster from here?

  1. Use a better algorithm
  2. Use SIMD
  3. Cache optimizations

Pretty graphs

Implementation, cpython 3.6, cython, pypy3, Rust Singlethreaded, Rust Multithreaded Runtime in seconds, 247.18, 114.3, 42, 4.95, 1.74

Python in Rust

$ Hello from Python 3.6.6 (default, Jun 27 2018, 13:11:40)

But why

Takeaways

  • Rust can make your Python fast
  • PyO3 makes integration easy

Thank you!

Sven-Hendrik Haase

https://svenstaro.org

svenstaro@gmail.com

You can hire me and my team

Slides and code at https://bit.ly/2Oiapp6