API

Parser

Interpreter

This is for interpreting and executing an executable document in JSON format. It can be called from the command line:

python3 interpreter.py [document parameters]

See README.md for more information.

Warning: eval and exec are used to run code in the document. Don’t execute documents that you haven’t verified yourself.

class pyla.interpreter.silent_list

A fake silent_list to prevent ReferenceErrors later.

class pyla.interpreter.MPLFigure

A fake MPLFigure to prevent ReferenceErrors later.

class pyla.interpreter.DataFrame

A fake DataFrame to prevent ReferenceErrors later.

class pyla.interpreter.CodeTimer

Context handler for timing code, use inside a with statement.

property duration_seconds

Calculate the duration (time between __enter__ and __exit__ in microseconds.

class pyla.interpreter.StdoutBuffer(buffer, encoding=None, errors=None, newline=None, line_buffering=False, write_through=False)

Used for capturing output to stdout.

write(string: Union[bytes, str]) → int

Write string to stream. Returns the number of characters written (which is always equal to the length of the string).

class pyla.interpreter.DocumentCompilationResult(parameters: List[stencila.schema.types.Parameter] = [], code: List[Union[pyla.parser.CodeChunkExecution, stencila.schema.types.CodeExpression]] = [])

Stores references to Parameters and Code that is parsed from a Document.

parameters: List[stencila.schema.types.Parameter]

Alias for field number 0

code: List[Union[pyla.parser.CodeChunkExecution, stencila.schema.types.CodeExpression]]

Alias for field number 1

class pyla.interpreter.DocumentCompiler

Parse an executable document (Article) and cache references to its parameters and code nodes.

compile(source: stencila.schema.types.Article) → pyla.interpreter.DocumentCompilationResult

Compile an Article by walking it and finding Parameters and CodeChunk or CodeExpression nodes.

These are set on a DocumentCompilationResult which can be passed to the Interpreter.

handle_item(item: Any, compilation_result: pyla.interpreter.DocumentCompilationResult) → None

Parse any kind of dict, list of Entity.

Returns nothing but updates the passed in DocumentCompilationResult.

static handle_code(item: Union[stencila.schema.types.CodeChunk, stencila.schema.types.CodeExpression], compilation_result: pyla.interpreter.DocumentCompilationResult) → None

Parse a CodeChunk or CodeExpression and add it to compilation_result.code list.

traverse_dict(_dict: dict, compilation_result: pyla.interpreter.DocumentCompilationResult) → None

Traverse into each item of a dict.

traverse_list(_list: List, compilation_result: pyla.interpreter.DocumentCompilationResult) → None

Traverse into each element in a list.

class pyla.interpreter.Interpreter

Execute a list of code blocks, maintaining its own globals scope for this execution run.

PROGRAMMING_LANGUAGES = ['py', 'python']

JSON Schema specification of the types of nodes that this intertpreter’s compile and execute methods are capable of handling

CODE_CAPABILITIES = {'properties': {'node': {'properties': {'programmingLanguage': {'enum': ['py', 'python']}, 'type': {'enum': ['CodeChunk', 'CodeExpression']}}, 'required': ['type', 'programmingLanguage'], 'type': 'object'}}, 'required': ['node'], 'type': 'object'}

The manifest of this interpreter’s capabilities and addresses.

Conforms to Executa’s Manifest interface.

static compile_code_chunk(chunk: stencila.schema.types.CodeChunk) → Tuple[pyla.parser.CodeChunkParseResult, stencila.schema.types.CodeChunk]

Compile a CodeChunk.

Returns a CodeChunkParseResult which is primarily needed for the AST, and the CodeChunk itself, which has its code metadata properties set.

static compile(node: Union[stencila.schema.types.Entity, int, float, bool, None, str, List[Any], Dict[str, Any]]) → Union[stencila.schema.types.Entity, int, float, bool, None, str, List[Any], Dict[str, Any]]

Compile a CodeChunk.

execute(node: Union[stencila.schema.types.Entity, int, float, bool, None, str, List[Any], Dict[str, Any]], parameter_values: Optional[Dict[str, Any]] = None) → Union[stencila.schema.types.Entity, int, float, bool, None, str, List[Any], Dict[str, Any]]

Execute a CodeChunk or CodeExpression.

static is_python_code(code: Union[stencila.schema.types.CodeChunk, stencila.schema.types.CodeExpression]) → bool

Is a CodeChunk or CodeExpression Python code?

execute_code_expression(expression: stencila.schema.types.CodeExpression, _locals: Dict[str, Any]) → stencila.schema.types.CodeExpression

Evaluate CodeExpression.text, and get the result. Catch any exception the occurs.

execute_code_chunk(chunk_execution: pyla.parser.CodeChunkExecution, _locals: Dict[str, Any]) → stencila.schema.types.CodeChunk

Execute a CodeChunk that has been parsed and stored in a CodeChunkExecution.

execute_statement(statement: _ast.stmt, chunk: stencila.schema.types.CodeChunk, _locals: Dict[str, Any], cc_outputs: List[str], duration: float) → Tuple[float, bool]

Execute a single AST statement.

The statement will be executed with eval or exec depending on its type (see parse_statement_runtime).

add_output(cc_outputs: List[Any], result: Any) → None

Add an output to cc_outputs.

Should be inside execute_statement as it’s only used there, but pylint complains about too many local variables.

static parse_statement_runtime(statement: _ast.stmt) → Tuple[bool, str, Callable]

Determine the information to execute a statement.

Returns the function with which to execute a statement (eval or exec), whether its output should be captured or not, and the compiled code to execute.

See the other comments below on how/why these functions are chosen.

static value_is_mpl(value: Any) → bool

Basic type checking to determine if a variable is a matplotlib figure.

static decode_mpl() → stencila.schema.types.ImageObject

Decode a matplotlib MPLFigure or MPLArtist into an ImageObject.

The matplotlib.pyplot.savefig function just saves the current MPL figure that’s in the context.

static decode_dataframe(data_frame: pyla.interpreter.DataFrame) → stencila.schema.types.Datatable

Decode a pandas DataFrame into a Datatable.

decode_output(output: Any) → Any

Check if the output is convertible from a special data type to a Stencila type.

If not, just return the original object.

class pyla.interpreter.MLPArtist

A fake MLPArtist to prevent ReferenceErrors later.

Servers

Module for server classes.

pyla.servers.rpc_json_object_encode(node: Union[Entity, int, float, bool, None, str, List[Any], Dict[str, Any]]) → Union[dict, str]

Like stencila.schema.json.object_encode but with support for JsonRpcError.

pyla.servers.to_json(node: Union[Entity, int, float, bool, None, str, List[Any], Dict[str, Any]]) → str

Convert a node including JsonRrpcErrors, to JSON.

pyla.servers.data_to_bytes(data: Any) → bytes

Convert data to bytes.

pyla.servers.encode_int(number: int) → bytes

Pack number into varint bytes.

pyla.servers.read_one(stream: Union[BinaryIO, socket.socket]) → int

Read a byte from a stream.

Raises EOFError if the stream ends while reading bytes.

pyla.servers.read_length_prefix(stream: Union[BinaryIO, socket.socket]) → int

Read a varint from stream.

pyla.servers.get_stream_buffer(stream: BinaryIO) → BinaryIO

Get the buffer from a stream, if it exists.

pyla.servers.io_read(stream: BinaryIO, count: int) → bytes

Read count bytes from stream or its underlying buffer (if it exists).

pyla.servers.io_write(stream: BinaryIO, message: bytes) → None

Write to a stream or its underlying buffer (if it exists).

pyla.servers.stream_read(stream: Union[BinaryIO, socket.socket], count: int) → bytes

Abstract reading from stream to work with IO (buffered/unbuffered) and sockets.

pyla.servers.stream_write(stream: Union[BinaryIO, socket.socket], message: bytes) → None

Abstract writing to stream to work with IO (buffered/unbuffered) and sockets.

pyla.servers.message_read(stream: Union[BinaryIO, socket.socket]) → str

Read a length-prefixed message from the stream.

pyla.servers.message_write(stream: Union[BinaryIO, socket.socket], message: str) → None

Write a length-prefixed message to the stream.

class pyla.servers.JsonRpcErrorCode(value)

Error codes defined in JSON-RPC 2.0

Codes -32000 to -32099 are reserved for implementation-defined server-errors.

Python implementation of Executa’s JsonRpcErrorCode.

ParseError = -32700

The JSON sent is not a valid Request object.

InvalidRequest = -32600

The method does not exist / is not available.

MethodNotFound = -32601

Invalid method parameter(s).

InvalidParams = -32602

Internal JSON-RPC error.

InternalError = -32603

Generic server error.

ServerError = -32000

Capability error.

exception pyla.servers.JsonRpcError(code: pyla.servers.JsonRpcErrorCode, message: str, data: Optional[Any] = None)

A JSON-RPC error that may be part of a response

Python implementation of Executa’s JsonRpcError.

code: pyla.servers.JsonRpcErrorCode

A primitive or structured value that contains additional information about the error. This may be omitted. The value of this member is defined by the server (e.g. detailed error information, nested errors etc.).

class pyla.servers.StreamServer(interpreter: pyla.interpreter.Interpreter, input_stream: Union[BinaryIO, socket.socket], output_stream: Union[BinaryIO, socket.socket])

A server that communicates using length-prefixed JSON-RPC messages over streams or sockets.

Python implementation of Executa’s StreamServer

read_message() → Iterable[str]

Read a length-prefixed message from the input stream then repeat.

write_message(message: str) → None

Write a length-prefixed message to the output stream.

receive_message(message: str) → str

Receive a JSON-RPC request and send back a JSON-RPC response.

The response may have a JSON-RPC error if the request was bad.

Python implementation of Executa’s Server.receive.

start() → None

Run the server in a loop forever.

This will run forever because the read_message generator never finishes.

class pyla.servers.StdioServer(interpreter: pyla.interpreter.Interpreter)

A StreamServer that uses stdio as the transport.

Python implementation of Executa’s StdioServer

System

Functions for system-related tasks (currently just registering/deregistering the executor manifest).

class pyla.system.ManifestManager

Register or deregister a manifest file.

static get_home_dir() → Optional[str]

Get the “home” directory of the current user, (or APPDATA dir on windows).

user_dir() → str

Get the current user’s Stencila data directory.

This is the directory that Stencila configuration settings, such as the installed Stencila hosts, and document buffers get stored.

manifest_dir() → str

Get the directory in which execution registration manifests are stored.

manifest_path() → str

Get the path the the execution registration manifest for this type of executor (Python).

register() → None

Write the registration manifest (MANIFEST) to the manifest path.

The command path (i.e. path to python) is set to the currently executing python binary, before right. This is to provide compatibility with virtual environments. The means that if you re-run the register command with different python binaries a different manifest will be written out.

deregister() → None

Remove the manifest file created with register.

Does not fail if the manifest file does not exist.

pyla.system.register() → None

Register the MANIFEST as defined by EXECUTORS_DIR_NAME and MANIFEST_FILE_NAME.

pyla.system.deregister() → None

Deregister the MANIFEST as defined by EXECUTORS_DIR_NAME and MANIFEST_FILE_NAME.