PowLu/TencentOS-tiny
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

micropython: add micropython component

Browse Source
This commit is contained in:
KY-zhang-X
2022-09-29 12:10:37 +08:00
parent 1514f1cb9b
commit dd76146324
2679 changed files with 354110 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

50
components/language/micropython/docs/reference/asm_thumb2_arith.rst Normal file
View File

@@ -0,0 +1,50 @@
Arithmetic instructions
=======================
Document conventions
--------------------
Notation: ``Rd, Rm, Rn`` denote ARM registers R0-R7. ``immN`` denotes an immediate
value having a width of N bits e.g. ``imm8``, ``imm3``. ``carry`` denotes
the carry condition flag, ``not(carry)`` denotes its complement. In the case of instructions
with more than one register argument, it is permissible for some to be identical. For example
the following will add the contents of R0 to itself, placing the result in R0:
* add(r0, r0, r0)
Arithmetic instructions affect the condition flags except where stated.
Addition
--------
* add(Rdn, imm8) ``Rdn = Rdn + imm8``
* add(Rd, Rn, imm3) ``Rd = Rn + imm3``
* add(Rd, Rn, Rm) ``Rd = Rn +Rm``
* adc(Rd, Rn) ``Rd = Rd + Rn + carry``
Subtraction
-----------
* sub(Rdn, imm8) ``Rdn = Rdn - imm8``
* sub(Rd, Rn, imm3) ``Rd = Rn - imm3``
* sub(Rd, Rn, Rm) ``Rd = Rn - Rm``
* sbc(Rd, Rn) ``Rd = Rd - Rn - not(carry)``
Negation
--------
* neg(Rd, Rn) ``Rd = -Rn``
Multiplication and division
---------------------------
* mul(Rd, Rn) ``Rd = Rd * Rn``
This produces a 32 bit result with overflow lost. The result may be treated as
signed or unsigned according to the definition of the operands.
* sdiv(Rd, Rn, Rm) ``Rd = Rn / Rm``
* udiv(Rd, Rn, Rm) ``Rd = Rn / Rm``
These functions perform signed and unsigned division respectively. Condition flags
are not affected.

90
components/language/micropython/docs/reference/asm_thumb2_compare.rst Normal file
View File

@@ -0,0 +1,90 @@
Comparison instructions
=======================
These perform an arithmetic or logical instruction on two arguments, discarding the result
but setting the condition flags. Typically these are used to test data values without changing
them prior to executing a conditional branch.
Document conventions
--------------------
Notation: ``Rd, Rm, Rn`` denote ARM registers R0-R7. ``imm8`` denotes an immediate
value having a width of 8 bits.
The Application Program Status Register (APSR)
----------------------------------------------
This contains four bits which are tested by the conditional branch instructions. Typically a
conditional branch will test multiple bits, for example ``bge(LABEL)``. The meaning of
condition codes can depend on whether the operands of an arithmetic instruction are viewed as
signed or unsigned integers. Thus ``bhi(LABEL)`` assumes unsigned numbers were processed while
``bgt(LABEL)`` assumes signed operands.
APSR Bits
---------
* Z (zero)
This is set if the result of an operation is zero or the operands of a comparison are equal.
* N (negative)
Set if the result is negative.
* C (carry)
An addition sets the carry flag when the result overflows out of the MSB, for example adding
0x80000000 and 0x80000000. By the nature of two's complement arithmetic this behaviour is reversed
on subtraction, with a borrow indicated by the carry bit being clear. Thus 0x10 - 0x01 is executed
as 0x10 + 0xffffffff which will set the carry bit.
* V (overflow)
The overflow flag is set if the result, viewed as a two's compliment number, has the "wrong" sign
in relation to the operands. For example adding 1 to 0x7fffffff will set the overflow bit because
the result (0x8000000), viewed as a two's complement integer, is negative. Note that in this instance
the carry bit is not set.
Comparison instructions
-----------------------
These set the APSR (Application Program Status Register) N (negative), Z (zero), C (carry) and V
(overflow) flags.
* cmp(Rn, imm8) ``Rn - imm8``
* cmp(Rn, Rm) ``Rn - Rm``
* cmn(Rn, Rm) ``Rn + Rm``
* tst(Rn, Rm) ``Rn & Rm``
Conditional execution
---------------------
The ``it`` and ``ite`` instructions provide a means of conditionally executing from one to four subsequent
instructions without the need for a label.
* it(<condition>) If then
Execute the next instruction if <condition> is true:
::
cmp(r0, r1)
it(eq)
mov(r0, 100) # runs if r0 == r1
# execution continues here
* ite(<condition>) If then else
If <condtion> is true, execute the next instruction, otherwise execute the
subsequent one. Thus:
::
cmp(r0, r1)
ite(eq)
mov(r0, 100) # runs if r0 == r1
mov(r0, 200) # runs if r0 != r1
# execution continues here
This may be extended to control the execution of upto four subsequent instructions: it[x[y[z]]]
where x,y,z=t/e; e.g. itt, itee, itete, ittte, itttt, iteee, etc.

36
components/language/micropython/docs/reference/asm_thumb2_directives.rst Normal file
View File

@@ -0,0 +1,36 @@
Assembler directives
====================
Labels
------
* label(INNER1)
This defines a label for use in a branch instruction. Thus elsewhere in the code a ``b(INNER1)``
will cause execution to continue with the instruction after the label directive.
Defining inline data
--------------------
The following assembler directives facilitate embedding data in an assembler code block.
* data(size, d0, d1 .. dn)
The data directive creates n array of data values in memory. The first argument specifies the
size in bytes of the subsequent arguments. Hence the first statement below will cause the
assembler to put three bytes (with values 2, 3 and 4) into consecutive memory locations
while the second will cause it to emit two four byte words.
::
data(1, 2, 3, 4)
data(4, 2, 100000)
Data values longer than a single byte are stored in memory in little-endian format.
* align(nBytes)
Align the following instruction to an nBytes value. ARM Thumb-2 instructions must be two
byte aligned, hence it's advisable to issue ``align(2)`` after ``data`` directives and
prior to any subsequent code. This ensures that the code will run irrespective of the
size of the data array.

77
components/language/micropython/docs/reference/asm_thumb2_float.rst Normal file
View File

@@ -0,0 +1,77 @@
Floating point instructions
===========================
These instructions support the use of the ARM floating point coprocessor
(on platforms such as the Pyboard which are equipped with one). The FPU
has 32 registers known as ``s0-s31`` each of which can hold a single
precision float. Data can be passed between the FPU registers and the
ARM core registers with the ``vmov`` instruction.
Note that MicroPython doesn't support passing floats to
assembler functions, nor can you put a float into ``r0`` and expect a
reasonable result. There are two ways to overcome this. The first is to
use arrays, and the second is to pass and/or return integers and convert
to and from floats in code.
Document conventions
--------------------
Notation: ``Sd, Sm, Sn`` denote FPU registers, ``Rd, Rm, Rn`` denote ARM core
registers. The latter can be any ARM core register although registers
``R13-R15`` are unlikely to be appropriate in this context.
Arithmetic
----------
* vadd(Sd, Sn, Sm) ``Sd = Sn + Sm``
* vsub(Sd, Sn, Sm) ``Sd = Sn - Sm``
* vneg(Sd, Sm) ``Sd = -Sm``
* vmul(Sd, Sn, Sm) ``Sd = Sn * Sm``
* vdiv(Sd, Sn, Sm) ``Sd = Sn / Sm``
* vsqrt(Sd, Sm) ``Sd = sqrt(Sm)``
Registers may be identical: ``vmul(S0, S0, S0)`` will execute ``S0 = S0*S0``
Move between ARM core and FPU registers
---------------------------------------
* vmov(Sd, Rm) ``Sd = Rm``
* vmov(Rd, Sm) ``Rd = Sm``
The FPU has a register known as FPSCR, similar to the ARM core's APSR, which stores condition
codes plus other data. The following instructions provide access to this.
* vmrs(APSR\_nzcv, FPSCR)
Move the floating-point N, Z, C, and V flags to the APSR N, Z, C, and V flags.
This is done after an instruction such as an FPU
comparison to enable the condition codes to be tested by the assembler
code. The following is a more general form of the instruction.
* vmrs(Rd, FPSCR) ``Rd = FPSCR``
Move between FPU register and memory
------------------------------------
* vldr(Sd, [Rn, offset]) ``Sd = [Rn + offset]``
* vstr(Sd, [Rn, offset]) ``[Rn + offset] = Sd``
Where ``[Rn + offset]`` denotes the memory address obtained by adding Rn to the offset. This
is specified in bytes. Since each float value occupies a 32 bit word, when accessing arrays of
floats the offset must always be a multiple of four bytes.
Data comparison
---------------
* vcmp(Sd, Sm)
Compare the values in Sd and Sm and set the FPU N, Z,
C, and V flags. This would normally be followed by ``vmrs(APSR_nzcv, FPSCR)``
to enable the results to be tested.
Convert between integer and float
---------------------------------
* vcvt\_f32\_s32(Sd, Sm) ``Sd = float(Sm)``
* vcvt\_s32\_f32(Sd, Sm) ``Sd = int(Sm)``

244
components/language/micropython/docs/reference/asm_thumb2_hints_tips.rst Normal file
View File

@@ -0,0 +1,244 @@
Hints and tips
==============
The following are some examples of the use of the inline assembler and some
information on how to work around its limitations. In this document the term
"assembler function" refers to a function declared in Python with the
``@micropython.asm_thumb`` decorator, whereas "subroutine" refers to assembler
code called from within an assembler function.
Code branches and subroutines
-----------------------------
It is important to appreciate that labels are local to an assembler function.
There is currently no way for a subroutine defined in one function to be called
from another.
To call a subroutine the instruction ``bl(LABEL)`` is issued. This transfers
control to the instruction following the ``label(LABEL)`` directive and stores
the return address in the link register (``lr`` or ``r14``). To return the
instruction ``bx(lr)`` is issued which causes execution to continue with
the instruction following the subroutine call. This mechanism implies that, if
a subroutine is to call another, it must save the link register prior to
the call and restore it before terminating.
The following rather contrived example illustrates a function call. Note that
it's necessary at the start to branch around all subroutine calls: subroutines
end execution with ``bx(lr)`` while the outer function simply "drops off the end"
in the style of Python functions.
::
@micropython.asm_thumb
def quad(r0):
b(START)
label(DOUBLE)
add(r0, r0, r0)
bx(lr)
label(START)
bl(DOUBLE)
bl(DOUBLE)
print(quad(10))
The following code example demonstrates a nested (recursive) call: the classic
Fibonacci sequence. Here, prior to a recursive call, the link register is saved
along with other registers which the program logic requires to be preserved.
::
@micropython.asm_thumb
def fib(r0):
b(START)
label(DOFIB)
push({r1, r2, lr})
cmp(r0, 1)
ble(FIBDONE)
sub(r0, 1)
mov(r2, r0) # r2 = n -1
bl(DOFIB)
mov(r1, r0) # r1 = fib(n -1)
sub(r0, r2, 1)
bl(DOFIB) # r0 = fib(n -2)
add(r0, r0, r1)
label(FIBDONE)
pop({r1, r2, lr})
bx(lr)
label(START)
bl(DOFIB)
for n in range(10):
print(fib(n))
Argument passing and return
---------------------------
The tutorial details the fact that assembler functions can support from zero to
three arguments, which must (if used) be named ``r0``, ``r1`` and ``r2``. When
the code executes the registers will be initialised to those values.
The data types which can be passed in this way are integers and memory
addresses. With current firmware all possible 32 bit values may be passed and
returned. If the return value may have the most significant bit set a Python
type hint should be employed to enable MicroPython to determine whether the
value should be interpreted as a signed or unsigned integer: types are
``int`` or ``uint``.
::
@micropython.asm_thumb
def uadd(r0, r1) -> uint:
add(r0, r0, r1)
``hex(uadd(0x40000000,0x40000000))`` will return 0x80000000, demonstrating the
passing and return of integers where bits 30 and 31 differ.
The limitations on the number of arguments and return values can be overcome by means
of the ``array`` module which enables any number of values of any type to be accessed.
Multiple arguments
~~~~~~~~~~~~~~~~~~
If a Python array of integers is passed as an argument to an assembler
function, the function will receive the address of a contiguous set of integers.
Thus multiple arguments can be passed as elements of a single array. Similarly a
function can return multiple values by assigning them to array elements.
Assembler functions have no means of determining the length of an array:
this will need to be passed to the function.
This use of arrays can be extended to enable more than three arrays to be used.
This is done using indirection: the ``uctypes`` module supports ``addressof()``
which will return the address of an array passed as its argument. Thus you can
populate an integer array with the addresses of other arrays:
::
from uctypes import addressof
@micropython.asm_thumb
def getindirect(r0):
ldr(r0, [r0, 0]) # Address of array loaded from passed array
ldr(r0, [r0, 4]) # Return element 1 of indirect array (24)
def testindirect():
a = array.array('i',[23, 24])
b = array.array('i',[0,0])
b[0] = addressof(a)
print(getindirect(b))
Non-integer data types
~~~~~~~~~~~~~~~~~~~~~~
These may be handled by means of arrays of the appropriate data type. For
example, single precision floating point data may be processed as follows.
This code example takes an array of floats and replaces its contents with
their squares.
::
from array import array
@micropython.asm_thumb
def square(r0, r1):
label(LOOP)
vldr(s0, [r0, 0])
vmul(s0, s0, s0)
vstr(s0, [r0, 0])
add(r0, 4)
sub(r1, 1)
bgt(LOOP)
a = array('f', (x for x in range(10)))
square(a, len(a))
print(a)
The uctypes module supports the use of data structures beyond simple
arrays. It enables a Python data structure to be mapped onto a bytearray
instance which may then be passed to the assembler function.
Named constants
---------------
Assembler code may be made more readable and maintainable by using named
constants rather than littering code with numbers. This may be achieved
thus:
::
MYDATA = const(33)
@micropython.asm_thumb
def foo():
mov(r0, MYDATA)
The const() construct causes MicroPython to replace the variable name
with its value at compile time. If constants are declared in an outer
Python scope they can be shared between multiple assembler functions and
with Python code.
Assembler code as class methods
-------------------------------
MicroPython passes the address of the object instance as the first argument
to class methods. This is normally of little use to an assembler function.
It can be avoided by declaring the function as a static method thus:
::
class foo:
@staticmethod
@micropython.asm_thumb
def bar(r0):
add(r0, r0, r0)
Use of unsupported instructions
-------------------------------
These can be coded using the data statement as shown below. While
``push()`` and ``pop()`` are supported the example below illustrates the
principle. The necessary machine code may be found in the ARM v7-M
Architecture Reference Manual. Note that the first argument of data
calls such as
::
data(2, 0xe92d, 0x0f00) # push r8,r9,r10,r11
indicates that each subsequent argument is a two byte quantity.
Overcoming MicroPython's integer restriction
--------------------------------------------
The Pyboard chip includes a CRC generator. Its use presents a problem in
MicroPython because the returned values cover the full gamut of 32 bit
quantities whereas small integers in MicroPython cannot have differing values
in bits 30 and 31. This limitation is overcome with the following code, which
uses assembler to put the result into an array and Python code to
coerce the result into an arbitrary precision unsigned integer.
::
from array import array
import stm
def enable_crc():
stm.mem32[stm.RCC + stm.RCC_AHB1ENR] |= 0x1000
def reset_crc():
stm.mem32[stm.CRC+stm.CRC_CR] = 1
@micropython.asm_thumb
def getval(r0, r1):
movwt(r3, stm.CRC + stm.CRC_DR)
str(r1, [r3, 0])
ldr(r2, [r3, 0])
str(r2, [r0, 0])
def getcrc(value):
a = array('i', [0])
getval(a, value)
return a[0] & 0xffffffff # coerce to arbitrary precision
enable_crc()
reset_crc()
for x in range(20):
print(hex(getcrc(0)))

73
components/language/micropython/docs/reference/asm_thumb2_index.rst Normal file
View File

@@ -0,0 +1,73 @@
.. _asm_thumb2_index:
Inline assembler for Thumb2 architectures
=========================================
This document assumes some familiarity with assembly language programming and should be read after studying
the :ref:`tutorial <pyboard_tutorial_assembler>`. For a detailed description of the instruction set consult the
Architecture Reference Manual detailed below.
The inline assembler supports a subset of the ARM Thumb-2 instruction set described here. The syntax tries
to be as close as possible to that defined in the above ARM manual, converted to Python function calls.
Instructions operate on 32 bit signed integer data except where stated otherwise. Most supported instructions
operate on registers ``R0-R7`` only: where ``R8-R15`` are supported this is stated. Registers ``R8-R12`` must be
restored to their initial value before return from a function. Registers ``R13-R15`` constitute the Link Register,
Stack Pointer and Program Counter respectively.
Document conventions
--------------------
Where possible the behaviour of each instruction is described in Python, for example
* add(Rd, Rn, Rm) ``Rd = Rn + Rm``
This enables the effect of instructions to be demonstrated in Python. In certain case this is impossible
because Python doesn't support concepts such as indirection. The pseudocode employed in such cases is
described on the relevant page.
Instruction categories
----------------------
The following sections details the subset of the ARM Thumb-2 instruction set supported by MicroPython.
.. toctree::
:maxdepth: 1
:numbered:
asm_thumb2_mov.rst
asm_thumb2_ldr.rst
asm_thumb2_str.rst
asm_thumb2_logical_bit.rst
asm_thumb2_arith.rst
asm_thumb2_compare.rst
asm_thumb2_label_branch.rst
asm_thumb2_stack.rst
asm_thumb2_misc.rst
asm_thumb2_float.rst
asm_thumb2_directives.rst
Usage examples
--------------
These sections provide further code examples and hints on the use of the assembler.
.. toctree::
:maxdepth: 1
:numbered:
asm_thumb2_hints_tips.rst
References
----------
- :ref:`Assembler Tutorial <pyboard_tutorial_assembler>`
- `Wiki hints and tips
<http://wiki.micropython.org/platforms/boards/pyboard/assembler>`__
- `uPy Inline Assembler source-code,
emitinlinethumb.c <https://github.com/micropython/micropython/blob/master/py/emitinlinethumb.c>`__
- `ARM Thumb2 Instruction Set Quick Reference
Card <http://infocenter.arm.com/help/topic/com.arm.doc.qrc0001l/QRC0001_UAL.pdf>`__
- `RM0090 Reference
Manual <http://www.google.ae/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&sqi=2&ved=0CBoQFjAA&url=http%3A%2F%2Fwww.st.com%2Fst-web-ui%2Fstatic%2Factive%2Fen%2Fresource%2Ftechnical%2Fdocument%2Freference_manual%2FDM00031020.pdf&ei=G0rSU66xFeuW0QWYwoD4CQ&usg=AFQjCNFuW6TgzE4QpahO_U7g3f3wdwecAg&sig2=iET-R0y9on_Pbflzf9aYDw&bvm=bv.71778758,bs.1,d.bGQ>`__
- ARM v7-M Architecture Reference Manual (Available on the
ARM site after a simple registration procedure. Also available on academic sites but beware of out of date versions.)

85
components/language/micropython/docs/reference/asm_thumb2_label_branch.rst Normal file
View File

@@ -0,0 +1,85 @@
Branch instructions
===================
These cause execution to jump to a target location usually specified by a label (see the ``label``
assembler directive). Conditional branches and the ``it`` and ``ite`` instructions test
the Application Program Status Register (APSR) N (negative), Z (zero), C (carry) and V
(overflow) flags to determine whether the branch should be executed.
Most of the exposed assembler instructions (including move operations) set the flags but
there are explicit comparison instructions to enable values to be tested.
Further detail on the meaning of the condition flags is provided in the section
describing comparison functions.
Document conventions
--------------------
Notation: ``Rm`` denotes ARM registers R0-R15. ``LABEL`` denotes a label defined with the
``label()`` assembler directive. ``<condition>`` indicates one of the following condition
specifiers:
* eq Equal to (result was zero)
* ne Not equal
* cs Carry set
* cc Carry clear
* mi Minus (negative)
* pl Plus (positive)
* vs Overflow set
* vc Overflow clear
* hi > (unsigned comparison)
* ls <= (unsigned comparison)
* ge >= (signed comparison)
* lt < (signed comparison)
* gt > (signed comparison)
* le <= (signed comparison)
Branch to label
---------------
* b(LABEL) Unconditional branch
* beq(LABEL) branch if equal
* bne(LABEL) branch if not equal
* bge(LABEL) branch if greater than or equal
* bgt(LABEL) branch if greater than
* blt(LABEL) branch if less than (<) (signed)
* ble(LABEL) branch if less than or equal to (<=) (signed)
* bcs(LABEL) branch if carry flag is set
* bcc(LABEL) branch if carry flag is clear
* bmi(LABEL) branch if negative
* bpl(LABEL) branch if positive
* bvs(LABEL) branch if overflow flag set
* bvc(LABEL) branch if overflow flag is clear
* bhi(LABEL) branch if higher (unsigned)
* bls(LABEL) branch if lower or equal (unsigned)
Long branches
-------------
The code produced by the branch instructions listed above uses a fixed bit width to specify the
branch destination, which is PC relative. Consequently in long programs where the
branch instruction is remote from its destination the assembler will produce a "branch not in
range" error. This can be overcome with the "wide" variants such as
* beq\_w(LABEL) long branch if equal
Wide branches use 4 bytes to encode the instruction (compared with 2 bytes for standard branch instructions).
Subroutines (functions)
-----------------------
When entering a subroutine the processor stores the return address in register r14, also
known as the link register (lr). Return to the instruction after the subroutine call is
performed by updating the program counter (r15 or pc) from the link register, This
process is handled by the following instructions.
* bl(LABEL)
Transfer execution to the instruction after ``LABEL`` storing the return address in
the link register (r14).
* bx(Rm) Branch to address specified by Rm.
Typically ``bx(lr)`` is issued to return from a subroutine. For nested subroutines the
link register of outer scopes must be saved (usually on the stack) before performing
inner subroutine calls.

23
components/language/micropython/docs/reference/asm_thumb2_ldr.rst Normal file
View File

@@ -0,0 +1,23 @@
Load register from memory
=========================
Document conventions
--------------------
Notation: ``Rt, Rn`` denote ARM registers R0-R7 except where stated. ``immN`` represents an immediate
value having a width of N bits hence ``imm5`` is constrained to the range 0-31. ``[Rn + immN]`` is the contents
of the memory address obtained by adding Rn and the offset ``immN``. Offsets are measured in
bytes. These instructions affect the condition flags.
Register Load
-------------
* ldr(Rt, [Rn, imm7]) ``Rt = [Rn + imm7]`` Load a 32 bit word
* ldrb(Rt, [Rn, imm5]) ``Rt = [Rn + imm5]`` Load a byte
* ldrh(Rt, [Rn, imm6]) ``Rt = [Rn + imm6]`` Load a 16 bit half word
Where a byte or half word is loaded, it is zero-extended to 32 bits.
The specified immediate offsets are measured in bytes. Hence in the case of ``ldr`` the 7 bit value
enables 32 bit word aligned values to be accessed with a maximum offset of 31 words. In the case of ``ldrh`` the
6 bit value enables 16 bit half-word aligned values to be accessed with a maximum offset of 31 half-words.

53
components/language/micropython/docs/reference/asm_thumb2_logical_bit.rst Normal file
View File

@@ -0,0 +1,53 @@
Logical & bitwise instructions
==============================
Document conventions
--------------------
Notation: ``Rd, Rn`` denote ARM registers R0-R7 except in the case of the
special instructions where R0-R15 may be used. ``Rn<a-b>`` denotes an ARM register
whose contents must lie in range ``a <= contents <= b``. In the case of instructions
with two register arguments, it is permissible for them to be identical. For example
the following will zero R0 (Python ``R0 ^= R0``) regardless of its initial contents.
* eor(r0, r0)
These instructions affect the condition flags except where stated.
Logical instructions
--------------------
* and\_(Rd, Rn) ``Rd &= Rn``
* orr(Rd, Rn) ``Rd |= Rn``
* eor(Rd, Rn) ``Rd ^= Rn``
* mvn(Rd, Rn) ``Rd = Rn ^ 0xffffffff`` i.e. Rd = 1's complement of Rn
* bic(Rd, Rn) ``Rd &= ~Rn`` bit clear Rd using mask in Rn
Note the use of "and\_" instead of "and", because "and" is a reserved keyword in Python.
Shift and rotation instructions
-------------------------------
* lsl(Rd, Rn<0-31>) ``Rd <<= Rn``
* lsr(Rd, Rn<1-32>) ``Rd = (Rd & 0xffffffff) >> Rn`` Logical shift right
* asr(Rd, Rn<1-32>) ``Rd >>= Rn`` arithmetic shift right
* ror(Rd, Rn<1-31>) ``Rd = rotate_right(Rd, Rn)`` Rd is rotated right Rn bits.
A rotation by (for example) three bits works as follows. If Rd initially
contains bits ``b31 b30..b0`` after rotation it will contain ``b2 b1 b0 b31 b30..b3``
Special instructions
--------------------
Condition codes are unaffected by these instructions.
* clz(Rd, Rn) ``Rd = count_leading_zeros(Rn)``
count_leading_zeros(Rn) returns the number of binary zero bits before the first binary one bit in Rn.
* rbit(Rd, Rn) ``Rd = bit_reverse(Rn)``
bit_reverse(Rn) returns the bit-reversed contents of Rn. If Rn contains bits ``b31 b30..b0`` Rd will be set
to ``b0 b1 b2..b31``
Trailing zeros may be counted by performing a bit reverse prior to executing clz.

13
components/language/micropython/docs/reference/asm_thumb2_misc.rst Normal file
View File

@@ -0,0 +1,13 @@
Miscellaneous instructions
==========================
* nop() ``pass`` no operation.
* wfi() Suspend execution in a low power state until an interrupt occurs.
* cpsid(flags) set the Priority Mask Register - disable interrupts.
* cpsie(flags) clear the Priority Mask Register - enable interrupts.
* mrs(Rd, special_reg) ``Rd = special_reg`` copy a special register to a general register. The special register
may be IPSR (Interrupt Status Register) or BASEPRI (Base Priority Register). The IPSR provides a means of determining
the exception number of an interrupt being processed. It contains zero if no interrupt is being processed.
Currently the ``cpsie()`` and ``cpsid()`` functions are partially implemented.
They require but ignore the flags argument and serve as a means of enabling and disabling interrupts.

27
components/language/micropython/docs/reference/asm_thumb2_mov.rst Normal file
View File

@@ -0,0 +1,27 @@
Register move instructions
==========================
Document conventions
--------------------
Notation: ``Rd, Rn`` denote ARM registers R0-R15. ``immN`` denotes an immediate
value having a width of N bits. These instructions affect the condition flags.
Register moves
--------------
Where immediate values are used, these are zero-extended to 32 bits. Thus
``mov(R0, 0xff)`` will set R0 to 255.
* mov(Rd, imm8) ``Rd = imm8``
* mov(Rd, Rn) ``Rd = Rn``
* movw(Rd, imm16) ``Rd = imm16``
* movt(Rd, imm16) ``Rd = (Rd & 0xffff) | (imm16 << 16)``
movt writes an immediate value to the top halfword of the destination register.
It does not affect the contents of the bottom halfword.
* movwt(Rd, imm32) ``Rd = imm32``
movwt is a pseudo-instruction: the MicroPython assembler emits a ``movw`` followed
by a ``movt`` to move a 32-bit value into Rd.

20
components/language/micropython/docs/reference/asm_thumb2_stack.rst Normal file
View File

@@ -0,0 +1,20 @@
Stack push and pop
==================
Document conventions
--------------------
The ``push()`` and ``pop()`` instructions accept as their argument a register set containing
a subset, or possibly all, of the general-purpose registers R0-R12 and the link register (lr or R14).
As with any Python set the order in which the registers are specified is immaterial. Thus the
in the following example the pop() instruction would restore R1, R7 and R8 to their contents prior
to the push():
* push({r1, r8, r7}) Save three registers on the stack.
* pop({r7, r1, r8}) Restore them
Stack operations
----------------
* push({regset}) Push a set of registers onto the stack
* pop({regset}) Restore a set of registers from the stack

21
components/language/micropython/docs/reference/asm_thumb2_str.rst Normal file
View File

@@ -0,0 +1,21 @@
Store register to memory
========================
Document conventions
--------------------
Notation: ``Rt, Rn`` denote ARM registers R0-R7 except where stated. ``immN`` represents an immediate
value having a width of N bits hence ``imm5`` is constrained to the range 0-31. ``[Rn + imm5]`` is the
contents of the memory address obtained by adding Rn and the offset ``imm5``. Offsets are measured in
bytes. These instructions do not affect the condition flags.
Register Store
--------------
* str(Rt, [Rn, imm7]) ``[Rn + imm7] = Rt`` Store a 32 bit word
* strb(Rt, [Rn, imm5]) ``[Rn + imm5] = Rt`` Store a byte (b0-b7)
* strh(Rt, [Rn, imm6]) ``[Rn + imm6] = Rt`` Store a 16 bit half word (b0-b15)
The specified immediate offsets are measured in bytes. Hence in the case of ``str`` the 7 bit value
enables 32 bit word aligned values to be accessed with a maximum offset of 31 words. In the case of ``strh`` the
6 bit value enables 16 bit half-word aligned values to be accessed with a maximum offset of 31 half-words.

462
components/language/micropython/docs/reference/constrained.rst Normal file
View File

@@ -0,0 +1,462 @@
.. _constrained:
MicroPython on microcontrollers
===============================
MicroPython is designed to be capable of running on microcontrollers. These
have hardware limitations which may be unfamiliar to programmers more familiar
with conventional computers. In particular the amount of RAM and nonvolatile
"disk" (flash memory) storage is limited. This tutorial offers ways to make
the most of the limited resources. Because MicroPython runs on controllers
based on a variety of architectures, the methods presented are generic: in some
cases it will be necessary to obtain detailed information from platform specific
documentation.
Flash memory
------------
On the Pyboard the simple way to address the limited capacity is to fit a micro
SD card. In some cases this is impractical, either because the device does not
have an SD card slot or for reasons of cost or power consumption; hence the
on-chip flash must be used. The firmware including the MicroPython subsystem is
stored in the onboard flash. The remaining capacity is available for use. For
reasons connected with the physical architecture of the flash memory part of
this capacity may be inaccessible as a filesystem. In such cases this space may
be employed by incorporating user modules into a firmware build which is then
flashed to the device.
There are two ways to achieve this: frozen modules and frozen bytecode. Frozen
modules store the Python source with the firmware. Frozen bytecode uses the
cross compiler to convert the source to bytecode which is then stored with the
firmware. In either case the module may be accessed with an import statement:
.. code::
import mymodule
The procedure for producing frozen modules and bytecode is platform dependent;
instructions for building the firmware can be found in the README files in the
relevant part of the source tree.
In general terms the steps are as follows:
* Clone the MicroPython `repository <https://github.com/micropython/micropython>`_.
* Acquire the (platform specific) toolchain to build the firmware.
* Build the cross compiler.
* Place the modules to be frozen in a specified directory (dependent on whether
the module is to be frozen as source or as bytecode).
* Build the firmware. A specific command may be required to build frozen
code of either type - see the platform documentation.
* Flash the firmware to the device.
RAM
---
When reducing RAM usage there are two phases to consider: compilation and
execution. In addition to memory consumption, there is also an issue known as
heap fragmentation. In general terms it is best to minimise the repeated
creation and destruction of objects. The reason for this is covered in the
section covering the `heap`_.
Compilation phase
~~~~~~~~~~~~~~~~~
When a module is imported, MicroPython compiles the code to bytecode which is
then executed by the MicroPython virtual machine (VM). The bytecode is stored
in RAM. The compiler itself requires RAM, but this becomes available for use
when the compilation has completed.
If a number of modules have already been imported the situation can arise where
there is insufficient RAM to run the compiler. In this case the import
statement will produce a memory exception.
If a module instantiates global objects on import it will consume RAM at the
time of import, which is then unavailable for the compiler to use on subsequent
imports. In general it is best to avoid code which runs on import; a better
approach is to have initialisation code which is run by the application after
all modules have been imported. This maximises the RAM available to the
compiler.
If RAM is still insufficient to compile all modules one solution is to
precompile modules. MicroPython has a cross compiler capable of compiling Python
modules to bytecode (see the README in the mpy-cross directory). The resulting
bytecode file has a .mpy extension; it may be copied to the filesystem and
imported in the usual way. Alternatively some or all modules may be implemented
as frozen bytecode: on most platforms this saves even more RAM as the bytecode
is run directly from flash rather than being stored in RAM.
Execution phase
~~~~~~~~~~~~~~~
There are a number of coding techniques for reducing RAM usage.
**Constants**
MicroPython provides a ``const`` keyword which may be used as follows:
.. code::
from micropython import const
ROWS = const(33)
_COLS = const(0x10)
a = ROWS
b = _COLS
In both instances where the constant is assigned to a variable the compiler
will avoid coding a lookup to the name of the constant by substituting its
literal value. This saves bytecode and hence RAM. However the ``ROWS`` value
will occupy at least two machine words, one each for the key and value in the
globals dictionary. The presence in the dictionary is necessary because another
module might import or use it. This RAM can be saved by prepending the name
with an underscore as in ``_COLS``: this symbol is not visible outside the
module so will not occupy RAM.
The argument to ``const()`` may be anything which, at compile time, evaluates
to a constant e.g. ``0x100``, ``1 << 8`` or ``(True, "string", b"bytes")``
(see section below for details). It can even include other const
symbols that have already been defined, e.g. ``1 << BIT``.
**Constant data structures**
Where there is a substantial volume of constant data and the platform supports
execution from Flash, RAM may be saved as follows. The data should be located in
Python modules and frozen as bytecode. The data must be defined as `bytes`
objects. The compiler 'knows' that `bytes` objects are immutable and ensures
that the objects remain in flash memory rather than being copied to RAM. The
`struct` module can assist in converting between `bytes` types and other
Python built-in types.
When considering the implications of frozen bytecode, note that in Python
strings, floats, bytes, integers, complex numbers and tuples are immutable.
Accordingly these will be frozen into flash (for tuples, only if all their
elements are immutable). Thus, in the line
.. code::
mystring = "The quick brown fox"
the actual string "The quick brown fox" will reside in flash. At runtime a
reference to the string is assigned to the *variable* ``mystring``. The reference
occupies a single machine word. In principle a long integer could be used to
store constant data:
.. code::
bar = 0xDEADBEEF0000DEADBEEF
As in the string example, at runtime a reference to the arbitrarily large
integer is assigned to the variable ``bar``. That reference occupies a
single machine word.
Tuples of constant objects are themselves constant. Such constant tuples are
optimised by the compiler so they do not need to be created at runtime each time
they are used. For example:
.. code::
foo = (1, 2, 3, 4, 5, 6, 100000, ("string", b"bytes", False, True))
This entire tuple will exist as a single object (potentially in flash if the
code is frozen) and referenced each time it is needed.
**Needless object creation**
There are a number of situations where objects may unwittingly be created and
destroyed. This can reduce the usability of RAM through fragmentation. The
following sections discuss instances of this.
**String concatenation**
Consider the following code fragments which aim to produce constant strings:
.. code::
var = "foo" + "bar"
var1 = "foo" "bar"
var2 = """\
foo\
bar"""
Each produces the same outcome, however the first needlessly creates two string
objects at runtime, allocates more RAM for concatenation before producing the
third. The others perform the concatenation at compile time which is more
efficient, reducing fragmentation.
Where strings must be dynamically created before being fed to a stream such as
a file it will save RAM if this is done in a piecemeal fashion. Rather than
creating a large string object, create a substring and feed it to the stream
before dealing with the next.
The best way to create dynamic strings is by means of the string ``format()``
method:
.. code::
var = "Temperature {:5.2f} Pressure {:06d}\n".format(temp, press)
**Buffers**
When accessing devices such as instances of UART, I2C and SPI interfaces, using
pre-allocated buffers avoids the creation of needless objects. Consider these
two loops:
.. code::
while True:
var = spi.read(100)
# process data
buf = bytearray(100)
while True:
spi.readinto(buf)
# process data in buf
The first creates a buffer on each pass whereas the second re-uses a pre-allocated
buffer; this is both faster and more efficient in terms of memory fragmentation.
**Bytes are smaller than ints**
On most platforms an integer consumes four bytes. Consider the three calls to the
function ``foo()``:
.. code::
def foo(bar):
for x in bar:
print(x)
foo([1, 2, 0xff])
foo((1, 2, 0xff))
foo(b'\1\2\xff')
In the first call a `list` of integers is created in RAM each time the code is
executed. The second call creates a constant `tuple` object (a `tuple` containing
only constant objects) as part of the compilation phase, so it is only created
once and is more efficient than the `list`. The third call efficiently
creates a `bytes` object consuming the minimum amount of RAM. If the module
were frozen as bytecode, both the `tuple` and `bytes` object would reside in flash.
**Strings Versus Bytes**
Python3 introduced Unicode support. This introduced a distinction between a
string and an array of bytes. MicroPython ensures that Unicode strings take no
additional space so long as all characters in the string are ASCII (i.e. have
a value < 126). If values in the full 8-bit range are required `bytes` and
`bytearray` objects can be used to ensure that no additional space will be
required. Note that most string methods (e.g. :meth:`str.strip()`) apply also to `bytes`
instances so the process of eliminating Unicode can be painless.
.. code::
s = 'the quick brown fox' # A string instance
b = b'the quick brown fox' # A bytes instance
Where it is necessary to convert between strings and bytes the :meth:`str.encode`
and the :meth:`bytes.decode` methods can be used. Note that both strings and bytes
are immutable. Any operation which takes as input such an object and produces
another implies at least one RAM allocation to produce the result. In the
second line below a new bytes object is allocated. This would also occur if ``foo``
were a string.
.. code::
foo = b' empty whitespace'
foo = foo.lstrip()
**Runtime compiler execution**
The Python funcitons `eval` and `exec` invoke the compiler at runtime, which
requires significant amounts of RAM. Note that the ``pickle`` library from
`micropython-lib` employs `exec`. It may be more RAM efficient to use the
`json` library for object serialisation.
**Storing strings in flash**
Python strings are immutable hence have the potential to be stored in read only
memory. The compiler can place in flash strings defined in Python code. As with
frozen modules it is necessary to have a copy of the source tree on the PC and
the toolchain to build the firmware. The procedure will work even if the
modules have not been fully debugged, so long as they can be imported and run.
After importing the modules, execute:
.. code::
micropython.qstr_info(1)
Then copy and paste all the Q(xxx) lines into a text editor. Check for and
remove lines which are obviously invalid. Open the file qstrdefsport.h which
will be found in ports/stm32 (or the equivalent directory for the architecture in
use). Copy and paste the corrected lines at the end of the file. Save the file,
rebuild and flash the firmware. The outcome can be checked by importing the
modules and again issuing:
.. code::
micropython.qstr_info(1)
The Q(xxx) lines should be gone.
.. _heap:
The heap
--------
When a running program instantiates an object the necessary RAM is allocated
from a fixed size pool known as the heap. When the object goes out of scope (in
other words becomes inaccessible to code) the redundant object is known as
"garbage". A process known as "garbage collection" (GC) reclaims that memory,
returning it to the free heap. This process runs automatically, however it can
be invoked directly by issuing `gc.collect()`.
The discourse on this is somewhat involved. For a 'quick fix' issue the
following periodically:
.. code::
gc.collect()
gc.threshold(gc.mem_free() // 4 + gc.mem_alloc())
Fragmentation
~~~~~~~~~~~~~
Say a program creates an object ``foo``, then an object ``bar``. Subsequently
``foo`` goes out of scope but ``bar`` remains. The RAM used by ``foo`` will be
reclaimed by GC. However if ``bar`` was allocated to a higher address, the
RAM reclaimed from ``foo`` will only be of use for objects no bigger than
``foo``. In a complex or long running program the heap can become fragmented:
despite there being a substantial amount of RAM available, there is insufficient
contiguous space to allocate a particular object, and the program fails with a
memory error.
The techniques outlined above aim to minimise this. Where large permanent buffers
or other objects are required it is best to instantiate these early in the
process of program execution before fragmentation can occur. Further improvements
may be made by monitoring the state of the heap and by controlling GC; these are
outlined below.
Reporting
~~~~~~~~~
A number of library functions are available to report on memory allocation and
to control GC. These are to be found in the `gc` and `micropython` modules.
The following example may be pasted at the REPL (``ctrl e`` to enter paste mode,
``ctrl d`` to run it).
.. code::
import gc
import micropython
gc.collect()
micropython.mem_info()
print('-----------------------------')
print('Initial free: {} allocated: {}'.format(gc.mem_free(), gc.mem_alloc()))
def func():
a = bytearray(10000)
gc.collect()
print('Func definition: {} allocated: {}'.format(gc.mem_free(), gc.mem_alloc()))
func()
print('Func run free: {} allocated: {}'.format(gc.mem_free(), gc.mem_alloc()))
gc.collect()
print('Garbage collect free: {} allocated: {}'.format(gc.mem_free(), gc.mem_alloc()))
print('-----------------------------')
micropython.mem_info(1)
Methods employed above:
* `gc.collect()` Force a garbage collection. See footnote.
* `micropython.mem_info()` Print a summary of RAM utilisation.
* `gc.mem_free()` Return the free heap size in bytes.
* `gc.mem_alloc()` Return the number of bytes currently allocated.
* ``micropython.mem_info(1)`` Print a table of heap utilisation (detailed below).
The numbers produced are dependent on the platform, but it can be seen that
declaring the function uses a small amount of RAM in the form of bytecode
emitted by the compiler (the RAM used by the compiler has been reclaimed).
Running the function uses over 10KiB, but on return ``a`` is garbage because it
is out of scope and cannot be referenced. The final `gc.collect()` recovers
that memory.
The final output produced by ``micropython.mem_info(1)`` will vary in detail but
may be interpreted as follows:
====== =================
Symbol Meaning
====== =================
. free block
h head block
= tail block
m marked head block
T tuple
L list
D dict
F float
B byte code
M module
====== =================
Each letter represents a single block of memory, a block being 16 bytes. So each
line of the heap dump represents 0x400 bytes or 1KiB of RAM.
Control of garbage collection
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
A GC can be demanded at any time by issuing `gc.collect()`. It is advantageous
to do this at intervals, firstly to pre-empt fragmentation and secondly for
performance. A GC can take several milliseconds but is quicker when there is
little work to do (about 1ms on the Pyboard). An explicit call can minimise that
delay while ensuring it occurs at points in the program when it is acceptable.
Automatic GC is provoked under the following circumstances. When an attempt at
allocation fails, a GC is performed and the allocation re-tried. Only if this
fails is an exception raised. Secondly an automatic GC will be triggered if the
amount of free RAM falls below a threshold. This threshold can be adapted as
execution progresses:
.. code::
gc.collect()
gc.threshold(gc.mem_free() // 4 + gc.mem_alloc())
This will provoke a GC when more than 25% of the currently free heap becomes
occupied.
In general modules should instantiate data objects at runtime using constructors
or other initialisation functions. The reason is that if this occurs on
initialisation the compiler may be starved of RAM when subsequent modules are
imported. If modules do instantiate data on import then `gc.collect()` issued
after the import will ameliorate the problem.
String operations
-----------------
MicroPython handles strings in an efficient manner and understanding this can
help in designing applications to run on microcontrollers. When a module
is compiled, strings which occur multiple times are stored once only, a process
known as string interning. In MicroPython an interned string is known as a ``qstr``.
In a module imported normally that single instance will be located in RAM, but
as described above, in modules frozen as bytecode it will be located in flash.
String comparisons are also performed efficiently using hashing rather than
character by character. The penalty for using strings rather than integers may
hence be small both in terms of performance and RAM usage - a fact which may
come as a surprise to C programmers.
Postscript
----------
MicroPython passes, returns and (by default) copies objects by reference. A
reference occupies a single machine word so these processes are efficient in
RAM usage and speed.
Where variables are required whose size is neither a byte nor a machine word
there are standard libraries which can assist in storing these efficiently and
in performing conversions. See the `array`, `struct` and `uctypes`
modules.
Footnote: gc.collect() return value
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
On Unix and Windows platforms the `gc.collect()` method returns an integer
which signifies the number of distinct memory regions that were reclaimed in the
collection (more precisely, the number of heads that were turned into frees). For
efficiency reasons bare metal ports do not return this value.

303
components/language/micropython/docs/reference/filesystem.rst Normal file
View File

@@ -0,0 +1,303 @@
.. _filesystem:
Working with filesystems
========================
.. contents::
This tutorial describes how MicroPython provides an on-device filesystem,
allowing standard Python file I/O methods to be used with persistent storage.
MicroPython automatically creates a default configuration and auto-detects the
primary filesystem, so this tutorial will be mostly useful if you want to modify
the partitioning, filesystem type, or use custom block devices.
The filesystem is typically backed by internal flash memory on the device, but
can also use external flash, RAM, or a custom block device.
On some ports (e.g. STM32), the filesystem may also be available over USB MSC to
a host PC. :ref:`pyboard_py` also provides a way for the host PC to access to
the filesystem on all ports.
Note: This is mainly for use on bare-metal ports like STM32 and ESP32. On ports
with an operating system (e.g. the Unix port) the filesystem is provided by the
host OS.
VFS
---
MicroPython implements a Unix-like Virtual File System (VFS) layer. All mounted
filesystems are combined into a single virtual filesystem, starting at the root
``/``. Filesystems are mounted into directories in this structure, and at
startup the working directory is changed to where the primary filesystem is
mounted.
On STM32 / Pyboard, the internal flash is mounted at ``/flash``, and optionally
the SDCard at ``/sd``. On ESP8266/ESP32, the primary filesystem is mounted at
``/``.
Block devices
-------------
A block device is an instance of a class that implements the
:class:`os.AbstractBlockDev` protocol.
Built-in block devices
~~~~~~~~~~~~~~~~~~~~~~
Ports provide built-in block devices to access their primary flash.
On power-on, MicroPython will attempt to detect the filesystem on the default
flash and configure and mount it automatically. If no filesystem is found,
MicroPython will attempt to create a FAT filesystem spanning the entire flash.
Ports can also provide a mechanism to "factory reset" the primary flash, usually
by some combination of button presses at power on.
STM32 / Pyboard
...............
The :ref:`pyb.Flash <pyb.Flash>` class provides access to the internal flash. On some
boards which have larger external flash (e.g. Pyboard D), it will use that
instead. The ``start`` kwarg should always be specified, i.e.
``pyb.Flash(start=0)``.
Note: For backwards compatibility, when constructed with no arguments (i.e.
``pyb.Flash()``), it only implements the simple block interface and reflects the
virtual device presented to USB MSC (i.e. it includes a virtual partition table
at the start).
ESP8266
.......
The internal flash is exposed as a block device object which is created in the
``flashbdev`` module on start up. This object is by default added as a global
variable so it can usually be accessed simply as ``bdev``. This implements the
extended interface.
ESP32
.....
The :class:`esp32.Partition` class implements a block device for partitions
defined for the board. Like ESP8266, there is a global variable ``bdev`` which
points to the default partition. This implements the extended interface.
Custom block devices
~~~~~~~~~~~~~~~~~~~~
The following class implements a simple block device that stores its data in
RAM using a ``bytearray``::
class RAMBlockDev:
def __init__(self, block_size, num_blocks):
self.block_size = block_size
self.data = bytearray(block_size * num_blocks)
def readblocks(self, block_num, buf):
for i in range(len(buf)):
buf[i] = self.data[block_num * self.block_size + i]
def writeblocks(self, block_num, buf):
for i in range(len(buf)):
self.data[block_num * self.block_size + i] = buf[i]
def ioctl(self, op, arg):
if op == 4: # get number of blocks
return len(self.data) // self.block_size
if op == 5: # get block size
return self.block_size
It can be used as follows::
import os
bdev = RAMBlockDev(512, 50)
os.VfsFat.mkfs(bdev)
os.mount(bdev, '/ramdisk')
An example of a block device that supports both the simple and extended
interface (i.e. both signatures and behaviours of the
:meth:`os.AbstractBlockDev.readblocks` and
:meth:`os.AbstractBlockDev.writeblocks` methods) is::
class RAMBlockDev:
def __init__(self, block_size, num_blocks):
self.block_size = block_size
self.data = bytearray(block_size * num_blocks)
def readblocks(self, block_num, buf, offset=0):
addr = block_num * self.block_size + offset
for i in range(len(buf)):
buf[i] = self.data[addr + i]
def writeblocks(self, block_num, buf, offset=None):
if offset is None:
# do erase, then write
for i in range(len(buf) // self.block_size):
self.ioctl(6, block_num + i)
offset = 0
addr = block_num * self.block_size + offset
for i in range(len(buf)):
self.data[addr + i] = buf[i]
def ioctl(self, op, arg):
if op == 4: # block count
return len(self.data) // self.block_size
if op == 5: # block size
return self.block_size
if op == 6: # block erase
return 0
As it supports the extended interface, it can be used with :class:`littlefs
<os.VfsLfs2>`::
import os
bdev = RAMBlockDev(512, 50)
os.VfsLfs2.mkfs(bdev)
os.mount(bdev, '/ramdisk')
Once mounted, the filesystem (regardless of its type) can be used as it
normally would be used from Python code, for example::
with open('/ramdisk/hello.txt', 'w') as f:
f.write('Hello world')
print(open('/ramdisk/hello.txt').read())
Filesystems
-----------
MicroPython ports can provide implementations of :class:`FAT <os.VfsFat>`,
:class:`littlefs v1 <os.VfsLfs1>` and :class:`littlefs v2 <os.VfsLfs2>`.
The following table shows which filesystems are included in the firmware by
default for given port/board combinations, however they can be optionally
enabled in a custom firmware build.
==================== ===== =========== ===========
Board FAT littlefs v1 littlefs v2
==================== ===== =========== ===========
pyboard 1.0, 1.1, D Yes No Yes
Other STM32 Yes No No
ESP8266 (1M) No No Yes
ESP8266 (2M+) Yes No Yes
ESP32 Yes No Yes
==================== ===== =========== ===========
FAT
~~~
The main advantage of the FAT filesystem is that it can be accessed over USB MSC
on supported boards (e.g. STM32) without any additional drivers required on the
host PC.
However, FAT is not tolerant to power failure during writes and this can lead to
filesystem corruption. For applications that do not require USB MSC, it is
recommended to use littlefs instead.
To format the entire flash using FAT::
# ESP8266 and ESP32
import os
os.umount('/')
os.VfsFat.mkfs(bdev)
os.mount(bdev, '/')
# STM32
import os, pyb
os.umount('/flash')
os.VfsFat.mkfs(pyb.Flash(start=0))
os.mount(pyb.Flash(start=0), '/flash')
os.chdir('/flash')
Littlefs
~~~~~~~~
Littlefs_ is a filesystem designed for flash-based devices, and is much more
resistant to filesystem corruption.
.. note:: There are reports of littlefs v1 and v2 failing in certain
situations, for details see `littlefs issue 347`_ and
`littlefs issue 295`_.
To format the entire flash using littlefs v2::
# ESP8266 and ESP32
import os
os.umount('/')
os.VfsLfs2.mkfs(bdev)
os.mount(bdev, '/')
# STM32
import os, pyb
os.umount('/flash')
os.VfsLfs2.mkfs(pyb.Flash(start=0))
os.mount(pyb.Flash(start=0), '/flash')
os.chdir('/flash')
A littlefs filesystem can be still be accessed on a PC over USB MSC using the
`littlefs FUSE driver`_. Note that you must specify both the ``--block_size``
and ``--block_count`` options to override the defaults. For example (after
building the littlefs-fuse executable)::
$ ./lfs --block_size=4096 --block_count=512 -o allow_other /dev/sdb1 mnt
This will allow the board's littlefs filesystem to be accessed at the ``mnt``
directory. To get the correct values of ``block_size`` and ``block_count`` use::
import pyb
f = pyb.Flash(start=0)
f.ioctl(1, 1) # initialise flash in littlefs raw-block mode
block_count = f.ioctl(4, 0)
block_size = f.ioctl(5, 0)
.. _littlefs FUSE driver: https://github.com/littlefs-project/littlefs-fuse
.. _Littlefs: https://github.com/littlefs-project/littlefs
.. _littlefs issue 295: https://github.com/littlefs-project/littlefs/issues/295
.. _littlefs issue 347: https://github.com/littlefs-project/littlefs/issues/347
Hybrid (STM32)
~~~~~~~~~~~~~~
By using the ``start`` and ``len`` kwargs to :class:`pyb.Flash`, you can create
block devices spanning a subset of the flash device.
For example, to configure the first 256kiB as FAT (and available over USB MSC),
and the remainder as littlefs::
import os, pyb
os.umount('/flash')
p1 = pyb.Flash(start=0, len=256*1024)
p2 = pyb.Flash(start=256*1024)
os.VfsFat.mkfs(p1)
os.VfsLfs2.mkfs(p2)
os.mount(p1, '/flash')
os.mount(p2, '/data')
os.chdir('/flash')
This might be useful to make your Python files, configuration and other
rarely-modified content available over USB MSC, but allowing for frequently
changing application data to reside on littlefs with better resilience to power
failure, etc.
The partition at offset ``0`` will be mounted automatically (and the filesystem
type automatically detected), but you can add::
import os, pyb
p2 = pyb.Flash(start=256*1024)
os.mount(p2, '/data')
to ``boot.py`` to mount the data partition.
Hybrid (ESP32)
~~~~~~~~~~~~~~
On ESP32, if you build custom firmware, you can modify ``partitions.csv`` to
define an arbitrary partition layout.
At boot, the partition named "vfs" will be mounted at ``/`` by default, but any
additional partitions can be mounted in your ``boot.py`` using::
import esp32, os
p = esp32.Partition.find(esp32.Partition.TYPE_DATA, label='foo')
os.mount(p, '/foo')

200
components/language/micropython/docs/reference/glossary.rst Normal file
View File

@@ -0,0 +1,200 @@
Glossary
========
.. glossary::
baremetal
A system without a (full-fledged) operating system, for example an
:term:`MCU`-based system. When running on a baremetal system,
MicroPython effectively functions like a small operating system,
running user programs and providing a command interpreter
(:term:`REPL`).
buffer protocol
Any Python object that can be automatically converted into bytes, such
as ``bytes``, ``bytearray``, ``memoryview`` and ``str`` objects, which
all implement the "buffer protocol".
board
Typically this refers to a printed circuit board (PCB) containing a
:term:`microcontroller <MCU>` and supporting components.
MicroPython firmware is typically provided per-board, as the firmware
contains both MCU-specific functionality but also board-level
functionality such as drivers or pin names.
bytecode
A compact representation of a Python program that generated by
compiling the Python source code. This is what the VM actually
executes. Bytecode is typically generated automatically at runtime and
is invisible to the user. Note that while :term:`CPython` and
MicroPython both use bytecode, the format is different. You can also
pre-compile source code offline using the :term:`cross-compiler`.
callee-owned tuple
This is a MicroPython-specific construct where, for efficiency
reasons, some built-in functions or methods may re-use the same
underlying tuple object to return data. This avoids having to allocate
a new tuple for every call, and reduces heap fragmentation. Programs
should not hold references to callee-owned tuples and instead only
extract data from them (or make a copy).
CircuitPython
A variant of MicroPython developed by `Adafruit Industries
<https://circuitpython.org>`_.
CPython
CPython is the reference implementation of the Python programming
language, and the most well-known one. It is, however, one of many
implementations (including Jython, IronPython, PyPy, and MicroPython).
While MicroPython's implementation differs substantially from CPython,
it aims to maintain as much compatibility as possible.
cross-compiler
Also known as ``mpy-cross``. This tool runs on your PC and converts a
:term:`.py file` containing MicroPython code into a :term:`.mpy file`
containing MicroPython bytecode. This means it loads faster (the board
doesn't have to compile the code), and uses less space on flash (the
bytecode is more space efficient).
driver
A MicroPython library that implements support for a particular
component, such as a sensor or display.
FFI
Acronym for Foreign Function Interface. A mechanism used by the
:term:`MicroPython Unix port` to access operating system functionality.
This is not available on :term:`baremetal` ports.
filesystem
Most MicroPython ports and boards provide a filesystem stored in flash
that is available to user code via the standard Python file APIs such
as ``open()``. Some boards also make this internal filesystem
accessible to the host via USB mass-storage.
frozen module
A Python module that has been cross compiled and bundled into the
firmware image. This reduces RAM requirements as the code is executed
directly from flash.
Garbage Collector
A background process that runs in Python (and MicroPython) to reclaim
unused memory in the :term:`heap`.
GPIO
General-purpose input/output. The simplest means to control electrical
signals (commonly referred to as "pins") on a microcontroller. GPIO
typically allows pins to be either input or output, and to set or get
their digital value (logical "0" or "1"). MicroPython abstracts GPIO
access using the :class:`machine.Pin` and :class:`machine.Signal`
classes.
GPIO port
A group of :term:`GPIO` pins, usually based on hardware properties of
these pins (e.g. controllable by the same register).
heap
A region of RAM where MicroPython stores dynamic data. It is managed
automatically by the :term:`Garbage Collector`. Different MCUs and
boards have vastly different amounts of RAM available for the heap, so
this will affect how complex your program can be.
interned string
An optimisation used by MicroPython to improve the efficiency of
working with strings. An interned string is referenced by its (unique)
identity rather than its address and can therefore be quickly compared
just by its identifier. It also means that identical strings can be
de-duplicated in memory. String interning is almost always invisible to
the user.
MCU
Microcontroller. Microcontrollers usually have much less resources
than a desktop, laptop, or phone, but are smaller, cheaper and
require much less power. MicroPython is designed to be small and
optimized enough to run on an average modern microcontroller.
micropython-lib
MicroPython is (usually) distributed as a single executable/binary
file with just few builtin modules. There is no extensive standard
library comparable with :term:`CPython`'s. Instead, there is a related,
but separate project `micropython-lib
<https://github.com/micropython/micropython-lib>`_ which provides
implementations for many modules from CPython's standard library.
Some of the modules are are implemented in pure Python, and are able to
be used on all ports. However, the majority of these modules use
:term:`FFI` to access operating system functionality, and as such can
only be used on the :term:`MicroPython Unix port` (with limited support
for Windows).
Unlike the :term:`CPython` stdlib, micropython-lib modules are
intended to be installed individually - either using manual copying or
using :term:`upip`.
MicroPython port
MicroPython supports different :term:`boards <board>`, RTOSes, and
OSes, and can be relatively easily adapted to new systems. MicroPython
with support for a particular system is called a "port" to that
system. Different ports may have widely different functionality. This
documentation is intended to be a reference of the generic APIs
available across different ports ("MicroPython core"). Note that some
ports may still omit some APIs described here (e.g. due to resource
constraints). Any such differences, and port-specific extensions
beyond the MicroPython core functionality, would be described in the
separate port-specific documentation.
MicroPython Unix port
The unix port is one of the major :term:`MicroPython ports
<MicroPython port>`. It is intended to run on POSIX-compatible
operating systems, like Linux, MacOS, FreeBSD, Solaris, etc. It also
serves as the basis of Windows port. The Unix port is very useful for
quick development and testing of the MicroPython language and
machine-independent features. It can also function in a similar way to
:term:`CPython`'s ``python`` executable.
.mpy file
The output of the :term:`cross-compiler`. A compiled form of a
:term:`.py file` that contains MicroPython bytecode instead of Python
source code.
native
Usually refers to "native code", i.e. machine code for the target
microcontroller (such as ARM Thumb, Xtensa, x86/x64). The ``@native``
decorator can be applied to a MicroPython function to generate native
code instead of bytecode for that function, which will likely be
faster but use more RAM.
port
Usually short for :term:`MicroPython port`, but could also refer to
:term:`GPIO port`.
.py file
A file containing Python source code.
REPL
An acronym for "Read, Eval, Print, Loop". This is the interactive
Python prompt, useful for debugging or testing short snippets of code.
Most MicroPython boards make a REPL available over a UART, and this is
typically accessible on a host PC via USB.
stream
Also known as a "file-like object". A Python object which provides
sequential read-write access to the underlying data. A stream object
implements a corresponding interface, which consists of methods like
``read()``, ``write()``, ``readinto()``, ``seek()``, ``flush()``,
``close()``, etc. A stream is an important concept in MicroPython;
many I/O objects implement the stream interface, and thus can be used
consistently and interchangeably in different contexts. For more
information on streams in MicroPython, see the `io` module.
UART
Acronym for "Universal Asynchronous Receiver/Transmitter". This is a
peripheral that sends data over a pair of pins (TX & RX). Many boards
include a way to make at least one of the UARTs available to a host PC
as a serial port over USB.
upip
(Literally, "micro pip"). A package manager for MicroPython, inspired
by :term:`CPython`'s pip, but much smaller and with reduced
functionality.
upip runs both on the :term:`Unix port <MicroPython Unix port>` and on
:term:`baremetal` ports which offer filesystem and networking support.

33
components/language/micropython/docs/reference/index.rst Normal file
View File

@@ -0,0 +1,33 @@
MicroPython language and implementation
=======================================
MicroPython aims to implement the Python 3.4 standard (with selected
features from later versions) with respect to language syntax, and most
of the features of MicroPython are identical to those described by the
"Language Reference" documentation at
`docs.python.org <https://docs.python.org/3/reference/index.html>`_.
The MicroPython standard library is described in the
:ref:`corresponding chapter <micropython_lib>`. The :ref:`cpython_diffs`
chapter describes differences between MicroPython and CPython (which
mostly concern standard library and types, but also some language-level
features).
This chapter describes features and peculiarities of MicroPython
implementation and the best practices to use them.
.. toctree::
:maxdepth: 1
glossary.rst
repl.rst
mpremote.rst
mpyfiles.rst
isr_rules.rst
speed_python.rst
constrained.rst
manifest.rst
packages.rst
asm_thumb2_index.rst
filesystem.rst
pyboard.py.rst

414
components/language/micropython/docs/reference/isr_rules.rst Normal file
View File

@@ -0,0 +1,414 @@
.. _isr_rules:
Writing interrupt handlers
==========================
On suitable hardware MicroPython offers the ability to write interrupt handlers in Python. Interrupt handlers
- also known as interrupt service routines (ISR's) - are defined as callback functions. These are executed
in response to an event such as a timer trigger or a voltage change on a pin. Such events can occur at any point
in the execution of the program code. This carries significant consequences, some specific to the MicroPython
language. Others are common to all systems capable of responding to real time events. This document covers
the language specific issues first, followed by a brief introduction to real time programming for those new to it.
This introduction uses vague terms like "slow" or "as fast as possible". This is deliberate, as speeds are
application dependent. Acceptable durations for an ISR are dependent on the rate at which interrupts occur,
the nature of the main program, and the presence of other concurrent events.
Tips and recommended practices
------------------------------
This summarises the points detailed below and lists the principal recommendations for interrupt handler code.
* Keep the code as short and simple as possible.
* Avoid memory allocation: no appending to lists or insertion into dictionaries, no floating point.
* Consider using ``micropython.schedule`` to work around the above constraint.
* Where an ISR returns multiple bytes use a pre-allocated ``bytearray``. If multiple integers are to be
shared between an ISR and the main program consider an array (``array.array``).
* Where data is shared between the main program and an ISR, consider disabling interrupts prior to accessing
the data in the main program and re-enabling them immediately afterwards (see Critical Sections).
* Allocate an emergency exception buffer (see below).
MicroPython issues
------------------
The emergency exception buffer
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If an error occurs in an ISR, MicroPython is unable to produce an error report unless a special buffer is created
for the purpose. Debugging is simplified if the following code is included in any program using interrupts.
.. code:: python
import micropython
micropython.alloc_emergency_exception_buf(100)
The emergency exception buffer can only hold one exception stack trace. This means that if a second exception is
thrown during the handling of an exception while the heap is locked, that second exception's stack trace will
replace the original one - even if the second exception is cleanly handled. This can lead to confusing exception
messages if the buffer is later printed.
Simplicity
~~~~~~~~~~
For a variety of reasons it is important to keep ISR code as short and simple as possible. It should do only what
has to be done immediately after the event which caused it: operations which can be deferred should be delegated
to the main program loop. Typically an ISR will deal with the hardware device which caused the interrupt, making
it ready for the next interrupt to occur. It will communicate with the main loop by updating shared data to indicate
that the interrupt has occurred, and it will return. An ISR should return control to the main loop as quickly
as possible. This is not a specific MicroPython issue so is covered in more detail :ref:`below <ISR>`.
Communication between an ISR and the main program
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Normally an ISR needs to communicate with the main program. The simplest means of doing this is via one or more
shared data objects, either declared as global or shared via a class (see below). There are various restrictions
and hazards around doing this, which are covered in more detail below. Integers, ``bytes`` and ``bytearray`` objects
are commonly used for this purpose along with arrays (from the array module) which can store various data types.
The use of object methods as callbacks
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MicroPython supports this powerful technique which enables an ISR to share instance variables with the underlying
code. It also enables a class implementing a device driver to support multiple device instances. The following
example causes two LED's to flash at different rates.
.. code:: python
import pyb, micropython
micropython.alloc_emergency_exception_buf(100)
class Foo(object):
def __init__(self, timer, led):
self.led = led
timer.callback(self.cb)
def cb(self, tim):
self.led.toggle()
red = Foo(pyb.Timer(4, freq=1), pyb.LED(1))
green = Foo(pyb.Timer(2, freq=0.8), pyb.LED(2))
In this example the ``red`` instance associates timer 4 with LED 1: when a timer 4 interrupt occurs ``red.cb()``
is called causing LED 1 to change state. The ``green`` instance operates similarly: a timer 2 interrupt
results in the execution of ``green.cb()`` and toggles LED 2. The use of instance methods confers two
benefits. Firstly a single class enables code to be shared between multiple hardware instances. Secondly, as
a bound method the callback function's first argument is ``self``. This enables the callback to access instance
data and to save state between successive calls. For example, if the class above had a variable ``self.count``
set to zero in the constructor, ``cb()`` could increment the counter. The ``red`` and ``green`` instances would
then maintain independent counts of the number of times each LED had changed state.
Creation of Python objects
~~~~~~~~~~~~~~~~~~~~~~~~~~
ISR's cannot create instances of Python objects. This is because MicroPython needs to allocate memory for the
object from a store of free memory block called the heap. This is not permitted in an interrupt handler because
heap allocation is not re-entrant. In other words the interrupt might occur when the main program is part way
through performing an allocation - to maintain the integrity of the heap the interpreter disallows memory
allocations in ISR code.
A consequence of this is that ISR's can't use floating point arithmetic; this is because floats are Python objects. Similarly
an ISR can't append an item to a list. In practice it can be hard to determine exactly which code constructs will
attempt to perform memory allocation and provoke an error message: another reason for keeping ISR code short and simple.
One way to avoid this issue is for the ISR to use pre-allocated buffers. For example a class constructor
creates a ``bytearray`` instance and a boolean flag. The ISR method assigns data to locations in the buffer and sets
the flag. The memory allocation occurs in the main program code when the object is instantiated rather than in the ISR.
The MicroPython library I/O methods usually provide an option to use a pre-allocated buffer. For
example ``pyb.i2c.recv()`` can accept a mutable buffer as its first argument: this enables its use in an ISR.
A means of creating an object without employing a class or globals is as follows:
.. code:: python
def set_volume(t, buf=bytearray(3)):
buf[0] = 0xa5
buf[1] = t >> 4
buf[2] = 0x5a
return buf
The compiler instantiates the default ``buf`` argument when the function is
loaded for the first time (usually when the module it's in is imported).
An instance of object creation occurs when a reference to a bound method is
created. This means that an ISR cannot pass a bound method to a function. One
solution is to create a reference to the bound method in the class constructor
and to pass that reference in the ISR. For example:
.. code:: python
class Foo():
def __init__(self):
self.bar_ref = self.bar # Allocation occurs here
self.x = 0.1
tim = pyb.Timer(4)
tim.init(freq=2)
tim.callback(self.cb)
def bar(self, _):
self.x *= 1.2
print(self.x)
def cb(self, t):
# Passing self.bar would cause allocation.
micropython.schedule(self.bar_ref, 0)
Other techniques are to define and instantiate the method in the constructor
or to pass :meth:`Foo.bar` with the argument *self*.
Use of Python objects
~~~~~~~~~~~~~~~~~~~~~
A further restriction on objects arises because of the way Python works. When an ``import`` statement is executed the
Python code is compiled to bytecode, with one line of code typically mapping to multiple bytecodes. When the code
runs the interpreter reads each bytecode and executes it as a series of machine code instructions. Given that an
interrupt can occur at any time between machine code instructions, the original line of Python code may be only
partially executed. Consequently a Python object such as a set, list or dictionary modified in the main loop
may lack internal consistency at the moment the interrupt occurs.
A typical outcome is as follows. On rare occasions the ISR will run at the precise moment in time when the object
is partially updated. When the ISR tries to read the object, a crash results. Because such problems typically occur
on rare, random occasions they can be hard to diagnose. There are ways to circumvent this issue, described in
:ref:`Critical Sections <Critical>` below.
It is important to be clear about what constitutes the modification of an object. An alteration to a built-in type
such as a dictionary is problematic. Altering the contents of an array or bytearray is not. This is because bytes
or words are written as a single machine code instruction which is not interruptible: in the parlance of real time
programming the write is atomic. A user defined object might instantiate an integer, array or bytearray. It is valid
for both the main loop and the ISR to alter the contents of these.
MicroPython supports integers of arbitrary precision. Values between 2**30 -1 and -2**30 will be stored in
a single machine word. Larger values are stored as Python objects. Consequently changes to long integers cannot
be considered atomic. The use of long integers in ISR's is unsafe because memory allocation may be
attempted as the variable's value changes.
Overcoming the float limitation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In general it is best to avoid using floats in ISR code: hardware devices normally handle integers and conversion
to floats is normally done in the main loop. However there are a few DSP algorithms which require floating point.
On platforms with hardware floating point (such as the Pyboard) the inline ARM Thumb assembler can be used to work
round this limitation. This is because the processor stores float values in a machine word; values can be shared
between the ISR and main program code via an array of floats.
Using micropython.schedule
~~~~~~~~~~~~~~~~~~~~~~~~~~
This function enables an ISR to schedule a callback for execution "very soon". The callback is queued for
execution which will take place at a time when the heap is not locked. Hence it can create Python objects
and use floats. The callback is also guaranteed to run at a time when the main program has completed any
update of Python objects, so the callback will not encounter partially updated objects.
Typical usage is to handle sensor hardware. The ISR acquires data from the hardware and enables it to
issue a further interrupt. It then schedules a callback to process the data.
Scheduled callbacks should comply with the principles of interrupt handler design outlined below. This is to
avoid problems resulting from I/O activity and the modification of shared data which can arise in any code
which pre-empts the main program loop.
Execution time needs to be considered in relation to the frequency with which interrupts can occur. If an
interrupt occurs while the previous callback is executing, a further instance of the callback will be queued
for execution; this will run after the current instance has completed. A sustained high interrupt repetition
rate therefore carries a risk of unconstrained queue growth and eventual failure with a ``RuntimeError``.
If the callback to be passed to `schedule()` is a bound method, consider the
note in "Creation of Python objects".
Exceptions
----------
If an ISR raises an exception it will not propagate to the main loop. The interrupt will be disabled unless the
exception is handled by the ISR code.
Interfacing to uasyncio
-----------------------
When an ISR runs it can preempt the `uasyncio` scheduler. If the ISR performs a `uasyncio`
operation the scheduler's operation can be disrupted. This applies whether the interrupt is hard
or soft and also applies if the ISR has passed execution to another function via
`micropython.schedule`. In particular creating or cancelling tasks is invalid in an ISR context.
The safe way to interact with `uasyncio` is to implement a coroutine with synchronisation performed by
`uasyncio.ThreadSafeFlag`. The following fragment illustrates the creation of a task in response
to an interrupt:
.. code:: python
tsf = uasyncio.ThreadSafeFlag()
def isr(_): # Interrupt handler
tsf.set()
async def foo():
while True:
await tsf.wait()
uasyncio.create_task(bar())
In this example there will be a variable amount of latency between the execution of the ISR and the execution
of ``foo()``. This is inherent to cooperative scheduling. The maximum latency is application
and platform dependent but may typically be measured in tens of ms.
General issues
--------------
This is merely a brief introduction to the subject of real time programming. Beginners should note
that design errors in real time programs can lead to faults which are particularly hard to diagnose. This is because
they can occur rarely and at intervals which are essentially random. It is crucial to get the initial design right and
to anticipate issues before they arise. Both interrupt handlers and the main program need to be designed
with an appreciation of the following issues.
.. _ISR:
Interrupt handler design
~~~~~~~~~~~~~~~~~~~~~~~~
As mentioned above, ISR's should be designed to be as simple as possible. They should always return in a short,
predictable period of time. This is important because when the ISR is running, the main loop is not: inevitably
the main loop experiences pauses in its execution at random points in the code. Such pauses can be a source of hard
to diagnose bugs particularly if their duration is long or variable. In order to understand the implications of
ISR run time, a basic grasp of interrupt priorities is required.
Interrupts are organised according to a priority scheme. ISR code may itself be interrupted by a higher priority
interrupt. This has implications if the two interrupts share data (see Critical Sections below). If such an interrupt
occurs it interposes a delay into the ISR code. If a lower priority interrupt occurs while the ISR is running, it
will be delayed until the ISR is complete: if the delay is too long, the lower priority interrupt may fail. A
further issue with slow ISR's is the case where a second interrupt of the same type occurs during its execution.
The second interrupt will be handled on termination of the first. However if the rate of incoming interrupts
consistently exceeds the capacity of the ISR to service them the outcome will not be a happy one.
Consequently looping constructs should be avoided or minimised. I/O to devices other than to the interrupting device
should normally be avoided: I/O such as disk access, ``print`` statements and UART access is relatively slow, and
its duration may vary. A further issue here is that filesystem functions are not reentrant: using filesystem I/O
in an ISR and the main program would be hazardous. Crucially ISR code should not wait on an event. I/O is acceptable
if the code can be guaranteed to return in a predictable period, for example toggling a pin or LED. Accessing the
interrupting device via I2C or SPI may be necessary but the time taken for such accesses should be calculated or
measured and its impact on the application assessed.
There is usually a need to share data between the ISR and the main loop. This may be done either through global
variables or via class or instance variables. Variables are typically integer or boolean types, or integer or byte
arrays (a pre-allocated integer array offers faster access than a list). Where multiple values are modified by
the ISR it is necessary to consider the case where the interrupt occurs at a time when the main program has
accessed some, but not all, of the values. This can lead to inconsistencies.
Consider the following design. An ISR stores incoming data in a bytearray, then adds the number of bytes
received to an integer representing total bytes ready for processing. The main program reads the number of bytes,
processes the bytes, then clears down the number of bytes ready. This will work until an interrupt occurs just
after the main program has read the number of bytes. The ISR puts the added data into the buffer and updates
the number received, but the main program has already read the number, so processes the data originally received.
The newly arrived bytes are lost.
There are various ways of avoiding this hazard, the simplest being to use a circular buffer. If it is not possible
to use a structure with inherent thread safety other ways are described below.
Reentrancy
~~~~~~~~~~
A potential hazard may occur if a function or method is shared between the main program and one or more ISR's or
between multiple ISR's. The issue here is that the function may itself be interrupted and a further instance of
that function run. If this is to occur, the function must be designed to be reentrant. How this is done is an
advanced topic beyond the scope of this tutorial.
.. _Critical:
Critical sections
~~~~~~~~~~~~~~~~~
An example of a critical section of code is one which accesses more than one variable which can be affected by an ISR. If
the interrupt happens to occur between accesses to the individual variables, their values will be inconsistent. This is
an instance of a hazard known as a race condition: the ISR and the main program loop race to alter the variables. To
avoid inconsistency a means must be employed to ensure that the ISR does not alter the values for the duration of
the critical section. One way to achieve this is to issue ``pyb.disable_irq()`` before the start of the section, and
``pyb.enable_irq()`` at the end. Here is an example of this approach:
.. code:: python
import pyb, micropython, array
micropython.alloc_emergency_exception_buf(100)
class BoundsException(Exception):
pass
ARRAYSIZE = const(20)
index = 0
data = array.array('i', 0 for x in range(ARRAYSIZE))
def callback1(t):
global data, index
for x in range(5):
data[index] = pyb.rng() # simulate input
index += 1
if index >= ARRAYSIZE:
raise BoundsException('Array bounds exceeded')
tim4 = pyb.Timer(4, freq=100, callback=callback1)
for loop in range(1000):
if index > 0:
irq_state = pyb.disable_irq() # Start of critical section
for x in range(index):
print(data[x])
index = 0
pyb.enable_irq(irq_state) # End of critical section
print('loop {}'.format(loop))
pyb.delay(1)
tim4.callback(None)
A critical section can comprise a single line of code and a single variable. Consider the following code fragment.
.. code:: python
count = 0
def cb(): # An interrupt callback
count +=1
def main():
# Code to set up the interrupt callback omitted
while True:
count += 1
This example illustrates a subtle source of bugs. The line ``count += 1`` in the main loop carries a specific race
condition hazard known as a read-modify-write. This is a classic cause of bugs in real time systems. In the main loop
MicroPython reads the value of ``count``, adds 1 to it, and writes it back. On rare occasions the interrupt occurs
after the read and before the write. The interrupt modifies ``count`` but its change is overwritten by the main
loop when the ISR returns. In a real system this could lead to rare, unpredictable failures.
As mentioned above, care should be taken if an instance of a Python built in type is modified in the main code and
that instance is accessed in an ISR. The code performing the modification should be regarded as a critical
section to ensure that the instance is in a valid state when the ISR runs.
Particular care needs to be taken if a dataset is shared between different ISR's. The hazard here is that the higher
priority interrupt may occur when the lower priority one has partially updated the shared data. Dealing with this
situation is an advanced topic beyond the scope of this introduction other than to note that mutex objects described
below can sometimes be used.
Disabling interrupts for the duration of a critical section is the usual and simplest way to proceed, but it disables
all interrupts rather than merely the one with the potential to cause problems. It is generally undesirable to disable
an interrupt for long. In the case of timer interrupts it introduces variability to the time when a callback occurs.
In the case of device interrupts, it can lead to the device being serviced too late with possible loss of data or
overrun errors in the device hardware. Like ISR's, a critical section in the main code should have a short, predictable
duration.
An approach to dealing with critical sections which radically reduces the time for which interrupts are disabled is to
use an object termed a mutex (name derived from the notion of mutual exclusion). The main program locks the mutex
before running the critical section and unlocks it at the end. The ISR tests whether the mutex is locked. If it is,
it avoids the critical section and returns. The design challenge is defining what the ISR should do in the event
that access to the critical variables is denied. A simple example of a mutex may be found
`here <https://github.com/peterhinch/micropython-samples.git>`_. Note that the mutex code does disable interrupts,
but only for the duration of eight machine instructions: the benefit of this approach is that other interrupts are
virtually unaffected.
Interrupts and the REPL
~~~~~~~~~~~~~~~~~~~~~~~
Interrupt handlers, such as those associated with timers, can continue to run
after a program terminates. This may produce unexpected results where you might
have expected the object raising the callback to have gone out of scope. For
example on the Pyboard:
.. code:: python
def bar():
foo = pyb.Timer(2, freq=4, callback=lambda t: print('.', end=''))
bar()
This continues to run until the timer is explicitly disabled or the board is
reset with ``ctrl D``.

145
components/language/micropython/docs/reference/manifest.rst Normal file
View File

@@ -0,0 +1,145 @@
MicroPython manifest files
==========================
When building firmware for a device the following components are included in
the compilation process:
- the core MicroPython virtual machine and runtime
- port-specific system code and drivers to interface with the
microcontroller/device that the firmware is targeting
- standard built-in modules, like ``sys``
- extended built-in modules, like ``json`` and ``machine``
- extra modules written in C/C++
- extra modules written in Python
All the modules included in the firmware are available via ``import`` from
Python code. The extra modules written in Python that are included in a build
(the last point above) are called *frozen modules*, and are specified by a
``manifest.py`` file. Changing this manifest requires rebuilding the firmware.
It's also possible to add additional modules to the filesystem of the device
once it is up and running. Adding and removing modules to/from the filesystem
does not require rebuilding the firmware so is a simpler process than rebuilding
firmware. The benefit of using a manifest is that frozen modules are more
efficient: they are faster to import and take up less RAM once imported.
MicroPython manifest files are Python files and can contain arbitrary Python
code. There are also a set of commands (predefined functions) which are used
to specify the Python source files to include. These commands are described
below.
Freezing source code
--------------------
.. function:: freeze(path, script=None, opt=0)
Freeze the input specified by *path*, automatically determining its type. A
``.py`` script will be compiled to a ``.mpy`` first then frozen, and a
``.mpy`` file will be frozen directly.
*path* must be a directory, which is the base directory to begin searching
for files. When importing the resulting frozen modules, the name of the
module will start after *path*, i.e. *path* is excluded from the module
name.
If *path* is relative, it is resolved to the current ``manifest.py``. Use
``$(MPY_DIR)``, ``$(MPY_LIB_DIR)``, ``$(PORT_DIR)``, ``$(BOARD_DIR)`` if you
need to access specific paths.
If *script* is None, all files in *path* will be frozen.
If *script* is an iterable then ``freeze()`` is called on all items of the
iterable (with the same *path* and *opt* passed through).
If *script* is a string then it specifies the file or directory to freeze,
and can include extra directories before the file or last directory. The
file or directory will be searched for in *path*. If *script* is a
directory then all files in that directory will be frozen.
*opt* is the optimisation level to pass to mpy-cross when compiling ``.py``
to ``.mpy``. These levels are described in :func:`micropython.opt_level`.
.. function:: freeze_as_str(path)
Freeze the given *path* and all ``.py`` scripts within it as a string, which
will be compiled upon import.
.. function:: freeze_as_mpy(path, script=None, opt=0)
Freeze the input by first compiling the ``.py`` scripts to ``.mpy`` files,
then freezing the resulting ``.mpy`` files. See ``freeze()`` for further
details on the arguments.
.. function:: freeze_mpy(path, script=None, opt=0)
Freeze the input, which must be ``.mpy`` files that are frozen directly.
See ``freeze()`` for further details on the arguments.
Including other manifest files
------------------------------
.. function:: include(manifest, **kwargs)
Include another manifest.
The *manifest* argument can be a string (filename) or an iterable of
strings.
Relative paths are resolved with respect to the current manifest file.
Optional *kwargs* can be provided which will be available to the included
script via the *options* variable.
For example:
.. code-block:: python3
include("path.py", extra_features=True)
then in path.py:
.. code-block:: python3
options.defaults(standard_features=True)
# freeze minimal modules.
if options.standard_features:
# freeze standard modules.
if options.extra_features:
# freeze extra modules.
Examples
--------
To freeze a single file which is available as ``import mydriver``, use:
.. code-block:: python3
freeze(".", "mydriver.py")
To freeze a set of files which are available as ``import test1`` and
``import test2``, and which are compiled with optimisation level 3, use:
.. code-block:: python3
freeze("/path/to/tests", ("test1.py", "test2.py"), opt=3)
To freeze a module which can be imported as ``import mymodule``, use:
.. code-block:: python3
freeze(
"../relative/path",
(
"mymodule/__init__.py",
"mymodule/core.py",
"mymodule/extra.py",
),
)
To include a manifest from the MicroPython repository, use:
.. code-block:: python3
include("$(MPY_DIR)/extmod/uasyncio/manifest.py")

250
components/language/micropython/docs/reference/mpremote.rst Normal file
View File

@@ -0,0 +1,250 @@
MicroPython remote control: mpremote
====================================
The ``mpremote`` command line tool provides an integrated set of utilities to
remotely interact with and automate a MicroPython device over a serial
connection.
To use mpremote install it via ``pip``:
.. code-block:: bash
$ pip install mpremote
The simplest way to use this tool is just by invoking it without any arguments:
.. code-block:: bash
mpremote
This command automatically detects and connects to the first available serial
device and provides an interactive REPL. Serial ports are opened in exclusive
mode, so running a second (or third, etc) instance of ``mpremote`` will connect
to subsequent serial devices, if any are available.
Commands
--------
For REPL access, running ``mpremote`` without any arguments is usually all that
is needed. ``mpremote`` also supports a set of commands given at the command
line which will perform various actions on remote MicroPython devices.
The full list of supported commands are:
- connect to a specified device via a device-name shortcut:
.. code-block:: bash
$ mpremote <device-shortcut>
- connect to specified device via name:
.. code-block:: bash
$ mpremote connect <device>
``<device>`` may be one of:
- ``list``: list available devices
- ``auto``: connect to the first available device
- ``id:<serial>``: connect to the device with USB serial number
``<serial>`` (the second entry in the output from the ``connect list``
command)
- ``port:<path>``: connect to the device with the given path
- any valid device name/path, to connect to that device
- disconnect current device:
.. code-block:: bash
$ mpremote disconnect
After a disconnect, auto soft-reset is enabled.
- resume a previous ``mpremote`` session:
.. code-block:: bash
$ mpremote resume
This disables auto soft-reset.
- perform a soft-reset of the device:
.. code-block:: bash
$ mpremote soft-reset
This will clear out the Python heap and restart the interpreter. It also
disables auto soft-reset.
- enter the REPL on the connected device:
.. code-block:: bash
$ mpremote repl [options]
Options are:
- ``--capture <file>``, to capture output of the REPL session to the given
file
- ``--inject-code <string>``, to specify characters to inject at the REPL when
Ctrl-J is pressed
- ``--inject-file <file>``, to specify a file to inject at the REPL when
Ctrl-K is pressed
- evaluate and print the result of a Python expression:
.. code-block:: bash
$ mpremote eval <string>
- execute the given Python code:
.. code-block:: bash
$ mpremote exec <string>
- run a script from the local filesystem:
.. code-block:: bash
$ mpremote run <file>
- execute filesystem commands on the device:
.. code-block:: bash
$ mpremote fs <command>
``<command>`` may be:
- ``cat <file..>`` to show the contents of a file or files on the device
- ``ls`` to list the current directory
- ``ls <dirs...>`` to list the given directories
- ``cp [-r] <src...> <dest>`` to copy files; use ":" as a prefix to specify
a file on the device
- ``rm <src...>`` to remove files on the device
- ``mkdir <dirs...>`` to create directories on the device
- ``rmdir <dirs...>`` to remove directories on the device
- mount the local directory on the remote device:
.. code-block:: bash
$ mpremote mount [options] <local-dir>
During usage, Ctrl-D will soft-reboot and normally reconnect the mount automatically.
If the unit has a main.py running at startup however the remount cannot occur.
In this case a raw mode soft reboot can be used: Ctrl-A Ctrl-D to reboot,
then Ctrl-B to get back to normal repl at which point the mount will be ready.
Options are:
- ``-l``, ``--unsafe-links``: By default an error will be raised if the device
accesses a file or directory which is outside (up one or more directory levels) the
local directory that is mounted. This option disables this check for symbolic
links, allowing the device to follow symbolic links outside of the local directory.
- unmount the local directory from the remote device:
.. code-block:: bash
$ mpremote umount
Multiple commands can be specified and they will be run sequentially.
Auto connection and soft-reset
------------------------------
Connection and disconnection will be done automatically at the start and end of
the execution of the tool, if such commands are not explicitly given. Automatic
connection will search for the first available serial device. If no action is
specified then the REPL will be entered.
Once connected to a device, ``mpremote`` will automatically soft-reset the
device if needed. This clears the Python heap and restarts the interpreter,
making sure that subsequent Python code executes in a fresh environment. Auto
soft-reset is performed the first time one of the following commands are
executed: ``mount``, ``eval``, ``exec``, ``run``, ``fs``. After doing a
soft-reset for the first time, it will not be done again automatically, until a
``disconnect`` command is issued.
Auto soft-reset behaviour can be controlled by the ``resume`` command. And the
``soft-reset`` command can be used to perform an explicit soft reset.
Shortcuts
---------
Shortcuts can be defined using the macro system. Built-in shortcuts are::
- ``devs``: list available devices (shortcut for ``connect list``)
- ``a0``, ``a1``, ``a2``, ``a3``: connect to /dev/ttyACM?
- ``u0``, ``u1``, ``u2``, ``u3``: connect to /dev/ttyUSB?
- ``c0``, ``c1``, ``c2``, ``c3``: connect to COM?
- ``cat``, ``ls``, ``cp``, ``rm``, ``mkdir``, ``rmdir``, ``df``: filesystem
commands
- ``reset``: reset the device
- ``bootloader``: make the device enter its bootloader
Any user configuration, including user-defined shortcuts, can be placed in the file
``.config/mpremote/config.py``. For example:
.. code-block:: python3
commands = {
"c33": "connect id:334D335C3138",
"bl": "bootloader",
"double x=4": "eval x*2", # x is an argument, with default 4
"wl_scan": ["exec", """
import network
wl = network.WLAN()
wl.active(1)
for ap in wl.scan():
print(ap)
""",],
"test": ["mount", ".", "exec", "import test"],
}
Examples
--------
.. code-block:: bash
mpremote
mpremote a1
mpremote connect /dev/ttyUSB0 repl
mpremote ls
mpremote a1 ls
mpremote exec "import micropython; micropython.mem_info()"
mpremote eval 1/2 eval 3/4
mpremote mount .
mpremote mount . exec "import local_script"
mpremote ls
mpremote cat boot.py
mpremote cp :main.py .
mpremote cp main.py :
mpremote cp -r dir/ :

194
components/language/micropython/docs/reference/mpyfiles.rst Normal file
View File

@@ -0,0 +1,194 @@
.. _mpy_files:
MicroPython .mpy files
======================
MicroPython defines the concept of an .mpy file which is a binary container
file format that holds precompiled code, and which can be imported like a
normal .py module. The file ``foo.mpy`` can be imported via ``import foo``,
as long as ``foo.mpy`` can be found in the usual way by the import machinery.
Usually, each directory listed in ``sys.path`` is searched in order. When
searching a particular directory ``foo.py`` is looked for first and if that
is not found then ``foo.mpy`` is looked for, then the search continues in the
next directory if neither is found. As such, ``foo.py`` will take precedence
over ``foo.mpy``.
These .mpy files can contain bytecode which is usually generated from Python
source files (.py files) via the ``mpy-cross`` program. For some architectures
an .mpy file can also contain native machine code, which can be generated in
a variety of ways, most notably from C source code.
Versioning and compatibility of .mpy files
------------------------------------------
A given .mpy file may or may not be compatible with a given MicroPython system.
Compatibility is based on the following:
* Version of the .mpy file: the version of the file must match the version
supported by the system loading it.
* Small integer bits: the .mpy file will require a minimum number of bits in
a small integer and the system loading it must support at least this many
bits.
* Native architecture: if the .mpy file contains native machine code then
it will specify the architecture of that machine code and the system
loading it must support execution of that architecture's code.
If a MicroPython system supports importing .mpy files then the
``sys.implementation._mpy`` field will exist and return an integer which
encodes the version (lower 8 bits), features and native architecture.
Trying to import an .mpy file that fails one of the first four tests will
raise ``ValueError('incompatible .mpy file')``. Trying to import an .mpy
file that fails the native architecture test (if it contains native machine
code) will raise ``ValueError('incompatible .mpy arch')``.
If importing an .mpy file fails then try the following:
* Determine the .mpy version and flags supported by your MicroPython system
by executing::
import sys
sys_mpy = sys.implementation._mpy
arch = [None, 'x86', 'x64',
'armv6', 'armv6m', 'armv7m', 'armv7em', 'armv7emsp', 'armv7emdp',
'xtensa', 'xtensawin'][sys_mpy >> 10]
print('mpy version:', sys_mpy & 0xff)
print('mpy flags:', end='')
if arch:
print(' -march=' + arch, end='')
print()
* Check the validity of the .mpy file by inspecting the first two bytes of
the file. The first byte should be an uppercase 'M' and the second byte
will be the version number, which should match the system version from above.
If it doesn't match then rebuild the .mpy file.
* Check if the system .mpy version matches the version emitted by ``mpy-cross``
that was used to build the .mpy file, found by ``mpy-cross --version``.
If it doesn't match then recompile ``mpy-cross`` from the Git repository
checked out at the tag (or hash) reported by ``mpy-cross --version``.
* Make sure you are using the correct ``mpy-cross`` flags, found by the code
above, or by inspecting the ``MPY_CROSS_FLAGS`` Makefile variable for the
port that you are using.
The following table shows the correspondence between MicroPython release
and .mpy version.
=================== ============
MicroPython release .mpy version
=================== ============
v1.19 and up 6
v1.12 - v1.18 5
v1.11 4
v1.9.3 - v1.10 3
v1.9 - v1.9.2 2
v1.5.1 - v1.8.7 0
=================== ============
For completeness, the next table shows the Git commit of the main
MicroPython repository at which the .mpy version was changed.
=================== ========================================
.mpy version change Git commit
=================== ========================================
5 to 6 f2040bfc7ee033e48acef9f289790f3b4e6b74e5
4 to 5 5716c5cf65e9b2cb46c2906f40302401bdd27517
3 to 4 9a5f92ea72754c01cc03e5efcdfe94021120531e
2 to 3 ff93fd4f50321c6190e1659b19e64fef3045a484
1 to 2 dd11af209d226b7d18d5148b239662e30ed60bad
0 to 1 6a11048af1d01c78bdacddadd1b72dc7ba7c6478
initial version 0 d8c834c95d506db979ec871417de90b7951edc30
=================== ========================================
Binary encoding of .mpy files
-----------------------------
MicroPython .mpy files are a binary container format with code objects (bytecode
and native machine code) stored internally in a nested hierarchy. The code for
the outer module is stored first, and then its children follow. Each child may
have further children, for example in the case of a class having methods, or a
function defining a lambda or comprehension. To keep files small while still
providing a large range of possible values it uses the concept of a
variably-encoded-unsigned-integer (vuint) in many places. Similar to utf-8
encoding, this encoding stores 7 bits per byte with the 8th bit (MSB) set
if one or more bytes follow. The bits of the unsigned integer are stored
in the vuint in LSB form.
The top-level of an .mpy file consists of three parts:
* The header.
* The global qstr and constant tables.
* The raw-code for the outer scope of the module.
This outer scope is executed when the .mpy file is imported.
You can inspect the contents of a .mpy file by using ``mpy-tool.py``, for
example (run from the root of the main MicroPython repository)::
$ ./tools/mpy-tool.py -xd myfile.mpy
The header
~~~~~~~~~~
The .mpy header is:
====== ================================
size field
====== ================================
byte value 0x4d (ASCII 'M')
byte .mpy version number
byte feature flags
byte number of bits in a small int
====== ================================
The global qstr and constant tables
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
An .mpy file contains a single qstr table, and a single constant object table.
These are global to the .mpy file, they are referenced by all nested raw-code
objects. The qstr table maps internal qstr number (internal to the .mpy file)
to the resolved qstr number of the runtime that the .mpy file is imported into.
This links the .mpy file with the rest of the system that it executes within.
The constant object table is populated with references to all constant objects
that the .mpy file needs.
====== ================================
size field
====== ================================
vuint number of qstrs
vuint number of constant objects
... qstr data
... encoded constant objects
====== ================================
Raw code elements
~~~~~~~~~~~~~~~~~
A raw-code element contains code, either bytecode or native machine code. Its
contents are:
====== ================================
size field
====== ================================
vuint type, size and whether there are sub-raw-code elements
... code (bytecode or machine code)
vuint number of sub-raw-code elements (only if non-zero)
... sub-raw-code elements
====== ================================
The first vuint in a raw-code element encodes the type of code stored in this
element (the two least-significant bits), whether this raw-code has any
children (the third least-significant bit), and the length of the code that
follows (the amount of RAM to allocate for it).
Following the vuint comes the code itself. Unless the code type is viper code
with relocations, this code is constant data and does not need to be modified.
If this raw-code has any children (as indicated by a bit in the first vuint),
following the code comes a vuint counting the number of sub-raw-code elements.
Finally any sub-raw-code elements are stored, recursively.

314
components/language/micropython/docs/reference/packages.rst Normal file
View File

@@ -0,0 +1,314 @@
.. _packages:
Distribution packages, package management, and deploying applications
=====================================================================
Just as the "big" Python, MicroPython supports creation of "third party"
packages, distributing them, and easily installing them in each user's
environment. This chapter discusses how these actions are achieved.
Some familiarity with Python packaging is recommended.
Overview
--------
Steps below represent a high-level workflow when creating and consuming
packages:
1. Python modules and packages are turned into distribution package
archives, and published at the Python Package Index (PyPI).
2. :term:`upip` package manager can be used to install a distribution package
on a :term:`MicroPython port` with networking capabilities (for example,
on the Unix port).
3. For ports without networking capabilities, an "installation image"
can be prepared on the Unix port, and transferred to a device by
suitable means.
4. For low-memory ports, the installation image can be frozen as the
bytecode into MicroPython executable, thus minimizing the memory
storage overheads.
The sections below describe this process in details.
Distribution packages
---------------------
Python modules and packages can be packaged into archives suitable for
transfer between systems, storing at the well-known location (PyPI),
and downloading on demand for deployment. These archives are known as
*distribution packages* (to differentiate them from Python packages
(means to organize Python source code)).
The MicroPython distribution package format is a well-known tar.gz
format, with some adaptations however. The Gzip compressor, used as
an external wrapper for TAR archives, by default uses 32KB dictionary
size, which means that to uncompress a compressed stream, 32KB of
contiguous memory needs to be allocated. This requirement may be not
satisfiable on low-memory devices, which may have total memory available
less than that amount, and even if not, a contiguous block like that
may be hard to allocate due to memory fragmentation. To accommodate
these constraints, MicroPython distribution packages use Gzip compression
with the dictionary size of 4K, which should be a suitable compromise
with still achieving some compression while being able to uncompressed
even by the smallest devices.
Besides the small compression dictionary size, MicroPython distribution
packages also have other optimizations, like removing any files from
the archive which aren't used by the installation process. In particular,
:term:`upip` package manager doesn't execute ``setup.py`` during installation
(see below), and thus that file is not included in the archive.
At the same time, these optimizations make MicroPython distribution
packages not compatible with :term:`CPython`'s package manager, ``pip``.
This isn't considered a big problem, because:
1. Packages can be installed with :term:`upip`, and then can be used with
CPython (if they are compatible with it).
2. In the other direction, majority of CPython packages would be
incompatible with MicroPython by various reasons, first of all,
the reliance on features not implemented by MicroPython.
Summing up, the MicroPython distribution package archives are highly
optimized for MicroPython's target environments, which are highly
resource constrained devices.
``upip`` package manager
------------------------
MicroPython distribution packages are intended to be installed using
the :term:`upip` package manager. :term:`upip` is a Python application which is
usually distributed (as frozen bytecode) with network-enabled
:term:`MicroPython ports <MicroPython port>`. At the very least,
:term:`upip` is available in the :term:`MicroPython Unix port`.
On any :term:`MicroPython port` providing :term:`upip`, it can be accessed as
following::
import upip
upip.help()
upip.install(package_or_package_list, [path])
Where *package_or_package_list* is the name of a distribution
package to install, or a list of such names to install multiple
packages. Optional *path* parameter specifies filesystem
location to install under and defaults to the standard library
location (see below).
An example of installing a specific package and then using it::
>>> import upip
>>> upip.install("micropython-pystone_lowmem")
[...]
>>> import pystone_lowmem
>>> pystone_lowmem.main()
Note that the name of Python package and the name of distribution
package for it in general don't have to match, and oftentimes they
don't. This is because PyPI provides a central package repository
for all different Python implementations and versions, and thus
distribution package names may need to be namespaced for a particular
implementation. For example, all packages from `micropython-lib`
follow this naming convention: for a Python module or package named
``foo``, the distribution package name is ``micropython-foo``.
For the ports which run MicroPython executable from the OS command
prompts (like the Unix port), `upip` can be (and indeed, usually is)
run from the command line instead of MicroPython's own REPL. The
commands which corresponds to the example above are::
micropython -m upip -h
micropython -m upip install [-p <path>] <packages>...
micropython -m upip install micropython-pystone_lowmem
[TODO: Describe installation path.]
Cross-installing packages
-------------------------
For :term:`MicroPython ports <MicroPython port>` without native networking
capabilities, the recommend process is "cross-installing" them into a
"directory image" using the :term:`MicroPython Unix port`, and then
transferring this image to a device by suitable means.
Installing to a directory image involves using ``-p`` switch to :term:`upip`::
micropython -m upip install -p install_dir micropython-pystone_lowmem
After this command, the package content (and contents of every dependency
packages) will be available in the ``install_dir/`` subdirectory. You
would need to transfer contents of this directory (without the
``install_dir/`` prefix) to the device, at the suitable location, where
it can be found by the Python ``import`` statement (see discussion of
the :term:`upip` installation path above).
Cross-installing packages with freezing
---------------------------------------
For the low-memory :term:`MicroPython ports <MicroPython port>`, the process
described in the previous section does not provide the most efficient
resource usage,because the packages are installed in the source form,
so need to be compiled to the bytecome on each import. This compilation
requires RAM, and the resulting bytecode is also stored in RAM, reducing
its amount available for storing application data. Moreover, the process
above requires presence of the filesystem on a device, and the most
resource-constrained devices may not even have it.
The bytecode freezing is a process which resolves all the issues
mentioned above:
* The source code is pre-compiled into bytecode and store as such.
* The bytecode is stored in ROM, not RAM.
* Filesystem is not required for frozen packages.
Using frozen bytecode requires building the executable (firmware)
for a given :term:`MicroPython port` from the C source code. Consequently,
the process is:
1. Follow the instructions for a particular port on setting up a
toolchain and building the port. For example, for ESP8266 port,
study instructions in ``ports/esp8266/README.md`` and follow them.
Make sure you can build the port and deploy the resulting
executable/firmware successfully before proceeding to the next steps.
2. Build :term:`MicroPython Unix port` and make sure it is in your PATH and
you can execute ``micropython``.
3. Change to port's directory (e.g. ``ports/esp8266/`` for ESP8266).
4. Run ``make clean-frozen``. This step cleans up any previous
modules which were installed for freezing (consequently, you need
to skip this step to add additional modules, instead of starting
from scratch).
5. Run ``micropython -m upip install -p modules <packages>...`` to
install packages you want to freeze.
6. Run ``make clean``.
7. Run ``make``.
After this, you should have the executable/firmware with modules as
the bytecode inside, which you can deploy the usual way.
Few notes:
1. Step 5 in the sequence above assumes that the distribution package
is available from PyPI. If that is not the case, you would need
to copy Python source files manually to ``modules/`` subdirectory
of the port directory. (Note that upip does not support
installing from e.g. version control repositories).
2. The firmware for baremetal devices usually has size restrictions,
so adding too many frozen modules may overflow it. Usually, you
would get a linking error if this happens. However, in some cases,
an image may be produced, which is not runnable on a device. Such
cases are in general bugs, and should be reported and further
investigated. If you face such a situation, as an initial step,
you may want to decrease the amount of frozen modules included.
Creating distribution packages
------------------------------
Distribution packages for MicroPython are created in the same manner
as for CPython or any other Python implementation, see references at
the end of chapter. Setuptools (instead of distutils) should be used,
because distutils do not support dependencies and other features. "Source
distribution" (``sdist``) format is used for packaging. The post-processing
discussed above, (and pre-processing discussed in the following section)
is achieved by using custom ``sdist`` command for setuptools. Thus, packaging
steps remain the same as for the standard setuptools, the user just
needs to override ``sdist`` command implementation by passing the
appropriate argument to ``setup()`` call::
from setuptools import setup
import sdist_upip
setup(
...,
cmdclass={'sdist': sdist_upip.sdist}
)
The sdist_upip.py module as referenced above can be found in
`micropython-lib`:
https://github.com/micropython/micropython-lib/blob/master/sdist_upip.py
Application resources
---------------------
A complete application, besides the source code, oftentimes also consists
of data files, e.g. web page templates, game images, etc. It's clear how
to deal with those when application is installed manually - you just put
those data files in the filesystem at some location and use the normal
file access functions.
The situation is different when deploying applications from packages - this
is more advanced, streamlined and flexible way, but also requires more
advanced approach to accessing data files. This approach is treating
the data files as "resources", and abstracting away access to them.
Python supports resource access using its "setuptools" library, using
``pkg_resources`` module. MicroPython, following its usual approach,
implements subset of the functionality of that module, specifically
``pkg_resources.resource_stream(package, resource)`` function.
The idea is that an application calls this function, passing a
resource identifier, which is a relative path to data file within
the specified package (usually top-level application package). It
returns a stream object which can be used to access resource contents.
Thus, the ``resource_stream()`` emulates interface of the standard
`open()` function.
Implementation-wise, ``resource_stream()`` uses file operations
underlyingly, if distribution package is install in the filesystem.
However, it also supports functioning without the underlying filesystem,
e.g. if the package is frozen as the bytecode. This however requires
an extra intermediate step when packaging application - creation of
"Python resource module".
The idea of this module is to convert binary data to a Python bytes
object, and put it into the dictionary, indexed by the resource name.
This conversion is done automatically using overridden ``sdist`` command
described in the previous section.
Let's trace the complete process using the following example. Suppose
your application has the following structure::
my_app/
__main__.py
utils.py
data/
page.html
image.png
``__main__.py`` and ``utils.py`` should access resources using the
following calls::
import pkg_resources
pkg_resources.resource_stream(__name__, "data/page.html")
pkg_resources.resource_stream(__name__, "data/image.png")
You can develop and debug using the :term:`MicroPython Unix port` as usual.
When time comes to make a distribution package out of it, just use
overridden "sdist" command from sdist_upip.py module as described in
the previous section.
This will create a Python resource module named ``R.py``, based on the
files declared in ``MANIFEST`` or ``MANIFEST.in`` files (any non-``.py``
file will be considered a resource and added to ``R.py``) - before
proceeding with the normal packaging steps.
Prepared like this, your application will work both when deployed to
filesystem and as frozen bytecode.
If you would like to debug ``R.py`` creation, you can run::
python3 setup.py sdist --manifest-only
Alternatively, you can use tools/mpy_bin2res.py script from the
MicroPython distribution, in which can you will need to pass paths
to all resource files::
mpy_bin2res.py data/page.html data/image.png
References
----------
* Python Packaging User Guide: https://packaging.python.org/
* Setuptools documentation: https://setuptools.readthedocs.io/
* Distutils documentation: https://docs.python.org/3/library/distutils.html

146
components/language/micropython/docs/reference/pyboard.py.rst Normal file
View File

@@ -0,0 +1,146 @@
.. _pyboard_py:
The pyboard.py tool
===================
This is a standalone Python tool that runs on your PC that provides a way to:
* Quickly run a Python script or command on a MicroPython device. This is useful
while developing MicroPython programs to quickly test code without needing to
copy files to/from the device.
* Access the filesystem on a device. This allows you to deploy your code to the
device (even if the board doesn't support USB MSC).
Despite the name, ``pyboard.py`` works on all MicroPython ports that support the
raw REPL (including STM32, ESP32, ESP8266, NRF).
You can download the latest version from `GitHub
<https://github.com/micropython/micropython/blob/master/tools/pyboard.py>`_. The
only dependency is the ``pyserial`` library which can be installed from PiPy or
your system package manager.
Running ``pyboard.py --help`` gives the following output:
.. code-block:: text
usage: pyboard [-h] [-d DEVICE] [-b BAUDRATE] [-u USER] [-p PASSWORD]
[-c COMMAND] [-w WAIT] [--follow | --no-follow] [-f]
[files [files ...]]
Run scripts on the pyboard.
positional arguments:
files input files
optional arguments:
-h, --help show this help message and exit
-d DEVICE, --device DEVICE
the serial device or the IP address of the pyboard
-b BAUDRATE, --baudrate BAUDRATE
the baud rate of the serial device
-u USER, --user USER the telnet login username
-p PASSWORD, --password PASSWORD
the telnet login password
-c COMMAND, --command COMMAND
program passed in as string
-w WAIT, --wait WAIT seconds to wait for USB connected board to become
available
--follow follow the output after running the scripts
[default if no scripts given]
-f, --filesystem perform a filesystem action: cp local :device | cp
:device local | cat path | ls [path] | rm path | mkdir
path | rmdir path
Running a command on the device
-------------------------------
This is useful for testing short snippets of code, or to script an interaction
with the device.::
$ pyboard.py --device /dev/ttyACM0 -c 'print(1+1)'
2
If you are often interacting with the same device, you can set the environment
variable ``PYBOARD_DEVICE`` as an alternative to using the ``--device``
command line option. For example, the following is equivalent to the previous
example::
$ export PYBOARD_DEVICE=/dev/ttyACM0
$ pyboard.py -c 'print(1+1)'
Similarly, the ``PYBOARD_BAUDRATE`` environment variable can be used
to set the default for the ``--baudrate`` option.
Running a script on the device
------------------------------
If you have a script, ``app.py`` that you want to run on a device, then use::
$ pyboard.py --device /dev/ttyACM0 app.py
Note that this doesn't actually copy app.py to the device's filesystem, it just
loads the code into RAM and executes it. Any output generated by the program
will be displayed.
If the program app.py does not finish then you'll need to stop ``pyboard.py``,
eg with Ctrl-C. The program ``app.py`` will still continue to run on the
MicroPython device.
Filesystem access
-----------------
Using the ``-f`` flag, the following filesystem operations are supported:
* ``cp src [src...] dest`` Copy files to/from the device.
* ``cat path`` Print the contents of a file on the device.
* ``ls [path]`` List contents of a directory (defaults to current working directory).
* ``rm path`` Remove a file.
* ``mkdir path`` Create a directory.
* ``rmdir path`` Remove a directory.
The ``cp`` command uses a ``ssh``-like convention for referring to local and
remote files. Any path starting with a ``:`` will be interpreted as on the
device, otherwise it will be local. So::
$ pyboard.py --device /dev/ttyACM0 -f cp main.py :main.py
will copy main.py from the current directory on the PC to a file named main.py
on the device. The filename can be omitted, e.g.::
$ pyboard.py --device /dev/ttyACM0 -f cp main.py :
is equivalent to the above.
Some more examples::
# Copy main.py from the device to the local PC.
$ pyboard.py --device /dev/ttyACM0 -f cp :main.py main.py
# Same, but using . instead.
$ pyboard.py --device /dev/ttyACM0 -f cp :main.py .
# Copy three files to the device, keeping their names
# and paths (note: `lib` must exist on the device)
$ pyboard.py --device /dev/ttyACM0 -f cp main.py app.py lib/foo.py :
# Remove a file from the device.
$ pyboard.py --device /dev/ttyACM0 -f rm util.py
# Print the contents of a file on the device.
$ pyboard.py --device /dev/ttyACM0 -f cat boot.py
...contents of boot.py...
Using the pyboard library
-------------------------
You can also use ``pyboard.py`` as a library for scripting interactions with a
MicroPython board.
.. code-block:: python
import pyboard
pyb = pyboard.Pyboard('/dev/ttyACM0', 115200)
pyb.enter_raw_repl()
ret = pyb.exec('print(1+1)')
print(ret)
pyb.exit_raw_repl()

300
components/language/micropython/docs/reference/repl.rst Normal file
View File

@@ -0,0 +1,300 @@
The MicroPython Interactive Interpreter Mode (aka REPL)
=======================================================
This section covers some characteristics of the MicroPython Interactive
Interpreter Mode. A commonly used term for this is REPL (read-eval-print-loop)
which will be used to refer to this interactive prompt.
Auto-indent
-----------
When typing python statements which end in a colon (for example if, for, while)
then the prompt will change to three dots (...) and the cursor will be indented
by 4 spaces. When you press return, the next line will continue at the same
level of indentation for regular statements or an additional level of indentation
where appropriate. If you press the backspace key then it will undo one
level of indentation.
If your cursor is all the way back at the beginning, pressing RETURN will then
execute the code that you've entered. The following shows what you'd see
after entering a for statement (the underscore shows where the cursor winds up):
>>> for i in range(30):
... _
If you then enter an if statement, an additional level of indentation will be
provided:
>>> for i in range(30):
... if i > 3:
... _
Now enter ``break`` followed by RETURN and press BACKSPACE:
>>> for i in range(30):
... if i > 3:
... break
... _
Finally type ``print(i)``, press RETURN, press BACKSPACE and press RETURN again:
>>> for i in range(30):
... if i > 3:
... break
... print(i)
...
0
1
2
3
>>>
Auto-indent won't be applied if the previous two lines were all spaces. This
means that you can finish entering a compound statement by pressing RETURN
twice, and then a third press will finish and execute.
Auto-completion
---------------
While typing a command at the REPL, if the line typed so far corresponds to
the beginning of the name of something, then pressing TAB will show
possible things that could be entered. For example, first import the machine
module by entering ``import machine`` and pressing RETURN.
Then type ``m`` and press TAB and it should expand to ``machine``.
Enter a dot ``.`` and press TAB again. You should see something like:
>>> machine.
__name__ info unique_id reset
bootloader freq rng idle
sleep deepsleep disable_irq enable_irq
Pin
The word will be expanded as much as possible until multiple possibilities exist.
For example, type ``machine.Pin.AF3`` and press TAB and it will expand to
``machine.Pin.AF3_TIM``. Pressing TAB a second time will show the possible
expansions:
>>> machine.Pin.AF3_TIM
AF3_TIM10 AF3_TIM11 AF3_TIM8 AF3_TIM9
>>> machine.Pin.AF3_TIM
Interrupting a running program
------------------------------
You can interrupt a running program by pressing Ctrl-C. This will raise a KeyboardInterrupt
which will bring you back to the REPL, providing your program doesn't intercept the
KeyboardInterrupt exception.
For example:
>>> for i in range(1000000):
... print(i)
...
0
1
2
3
...
6466
6467
6468
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
KeyboardInterrupt:
>>>
Paste mode
----------
If you want to paste some code into your terminal window, the auto-indent feature
will mess things up. For example, if you had the following python code: ::
def foo():
print('This is a test to show paste mode')
print('Here is a second line')
foo()
and you try to paste this into the normal REPL, then you will see something like
this:
>>> def foo():
... print('This is a test to show paste mode')
... print('Here is a second line')
... foo()
...
Traceback (most recent call last):
File "<stdin>", line 3
IndentationError: unexpected indent
If you press Ctrl-E, then you will enter paste mode, which essentially turns off
the auto-indent feature, and changes the prompt from ``>>>`` to ``===``. For example:
>>>
paste mode; Ctrl-C to cancel, Ctrl-D to finish
=== def foo():
=== print('This is a test to show paste mode')
=== print('Here is a second line')
=== foo()
===
This is a test to show paste mode
Here is a second line
>>>
Paste Mode allows blank lines to be pasted. The pasted text is compiled as if
it were a file. Pressing Ctrl-D exits paste mode and initiates the compilation.
Soft reset
----------
A soft reset will reset the python interpreter, but tries not to reset the
method by which you're connected to the MicroPython board (USB-serial, or Wifi).
You can perform a soft reset from the REPL by pressing Ctrl-D, or from your python
code by executing: ::
machine.soft_reset()
For example, if you reset your MicroPython board, and you execute a dir()
command, you'd see something like this:
>>> dir()
['__name__', 'pyb']
Now create some variables and repeat the dir() command:
>>> i = 1
>>> j = 23
>>> x = 'abc'
>>> dir()
['j', 'x', '__name__', 'pyb', 'i']
>>>
Now if you enter Ctrl-D, and repeat the dir() command, you'll see that your
variables no longer exist:
.. code-block:: python
MPY: sync filesystems
MPY: soft reboot
MicroPython v1.5-51-g6f70283-dirty on 2015-10-30; PYBv1.0 with STM32F405RG
Type "help()" for more information.
>>> dir()
['__name__', 'pyb']
>>>
The special variable _ (underscore)
-----------------------------------
When you use the REPL, you may perform computations and see the results.
MicroPython stores the results of the previous statement in the variable _ (underscore).
So you can use the underscore to save the result in a variable. For example:
>>> 1 + 2 + 3 + 4 + 5
15
>>> x = _
>>> x
15
>>>
Raw mode and raw-paste mode
---------------------------
Raw mode (also called raw REPL) is not something that a person would normally use.
It is intended for programmatic use and essentially behaves like paste mode with
echo turned off, and with optional flow control.
Raw mode is entered using Ctrl-A. You then send your python code, followed by
a Ctrl-D. The Ctrl-D will be acknowledged by 'OK' and then the python code will
be compiled and executed. Any output (or errors) will be sent back. Entering
Ctrl-B will leave raw mode and return the the regular (aka friendly) REPL.
Raw-paste mode is an additional mode within the raw REPL that includes flow control,
and which compiles code as it receives it. This makes it more robust for high-speed
transfer of code into the device, and it also uses less RAM when receiving because
it does not need to store a verbatim copy of the code before compiling (unlike
standard raw mode).
Raw-paste mode uses the following protocol:
#. Enter raw REPL as usual via ctrl-A.
#. Write 3 bytes: ``b"\x05A\x01"`` (ie ctrl-E then "A" then ctrl-A).
#. Read 2 bytes to determine if the device entered raw-paste mode:
* If the result is ``b"R\x00"`` then the device understands the command but
doesn't support raw paste.
* If the result is ``b"R\x01"`` then the device does support raw paste and
has entered this mode.
* Otherwise the result should be ``b"ra"`` and the device doesn't support raw
paste and the string ``b"w REPL; CTRL-B to exit\r\n>"`` should be read and
discarded.
#. If the device is in raw-paste mode then continue, otherwise fallback to
standard raw mode.
#. Read 2 bytes, this is the flow control window-size-increment (in bytes)
stored as a 16-bit unsigned little endian integer. The initial value for the
remaining-window-size variable should be set to this number.
#. Write out the code to the device:
* While there are bytes to send, write up to the remaining-window-size worth
of bytes, and decrease the remaining-window-size by the number of bytes
written.
* If the remaining-window-size is 0, or there is a byte waiting to read, read
1 byte. If this byte is ``b"\x01"`` then increase the remaining-window-size
by the window-size-increment from step 5. If this byte is ``b"\x04"`` then
the device wants to end the data reception, and ``b"\x04"`` should be
written to the device and no more code sent after that. (Note: if there is
a byte waiting to be read from the device then it does not need to be read
and acted upon immediately, the device will continue to consume incoming
bytes as long as reamining-window-size is greater than 0.)
#. When all code has been written to the device, write ``b"\x04"`` to indicate
end-of-data.
#. Read from the device until ``b"\x04"`` is received. At this point the device
has received and compiled all of the code that was sent and is executing it.
#. The device outputs any characters produced by the executing code. When (if)
the code finishes ``b"\x04"`` will be output, followed by any exception that
was uncaught, followed again by ``b"\x04"``. It then goes back to the
standard raw REPL and outputs ``b">"``.
For example, starting at a new line at the normal (friendly) REPL, if you write::
b"\x01\x05A\x01print(123)\x04"
Then the device will respond with something like::
b"\r\nraw REPL; CTRL-B to exit\r\n>R\x01\x80\x00\x01\x04123\r\n\x04\x04>"
Broken down over time this looks like::
# Step 1: enter raw REPL
write: b"\x01"
read: b"\r\nraw REPL; CTRL-B to exit\r\n>"
# Step 2-5: enter raw-paste mode
write: b"\x05A\x01"
read: b"R\x01\x80\x00\x01"
# Step 6-8: write out code
write: b"print(123)\x04"
read: b"\x04"
# Step 9: code executes and result is read
read: b"123\r\n\x04\x04>"
In this case the flow control window-size-increment is 128 and there are two
windows worth of data immediately available at the start, one from the initial
window-size-increment value and one from the explicit ``b"\x01"`` value that
is sent. So this means up to 256 bytes can be written to begin with before
waiting or checking for more incoming flow-control characters.
The ``tools/pyboard.py`` program uses the raw REPL, including raw-paste mode, to
execute Python code on a MicroPython-enabled board.

348
components/language/micropython/docs/reference/speed_python.rst Normal file
View File

@@ -0,0 +1,348 @@
.. _speed_python:
Maximising MicroPython speed
============================
.. contents::
This tutorial describes ways of improving the performance of MicroPython code.
Optimisations involving other languages are covered elsewhere, namely the use
of modules written in C and the MicroPython inline assembler.
The process of developing high performance code comprises the following stages
which should be performed in the order listed.
* Design for speed.
* Code and debug.
Optimisation steps:
* Identify the slowest section of code.
* Improve the efficiency of the Python code.
* Use the native code emitter.
* Use the viper code emitter.
* Use hardware-specific optimisations.
Designing for speed
-------------------
Performance issues should be considered at the outset. This involves taking a view
on the sections of code which are most performance critical and devoting particular
attention to their design. The process of optimisation begins when the code has
been tested: if the design is correct at the outset optimisation will be
straightforward and may actually be unnecessary.
Algorithms
~~~~~~~~~~
The most important aspect of designing any routine for performance is ensuring that
the best algorithm is employed. This is a topic for textbooks rather than for a
MicroPython guide but spectacular performance gains can sometimes be achieved
by adopting algorithms known for their efficiency.
RAM allocation
~~~~~~~~~~~~~~
To design efficient MicroPython code it is necessary to have an understanding of the
way the interpreter allocates RAM. When an object is created or grows in size
(for example where an item is appended to a list) the necessary RAM is allocated
from a block known as the heap. This takes a significant amount of time;
further it will on occasion trigger a process known as garbage collection which
can take several milliseconds.
Consequently the performance of a function or method can be improved if an object is created
once only and not permitted to grow in size. This implies that the object persists
for the duration of its use: typically it will be instantiated in a class constructor
and used in various methods.
This is covered in further detail :ref:`Controlling garbage collection <controlling_gc>` below.
Buffers
~~~~~~~
An example of the above is the common case where a buffer is required, such as one
used for communication with a device. A typical driver will create the buffer in the
constructor and use it in its I/O methods which will be called repeatedly.
The MicroPython libraries typically provide support for pre-allocated buffers. For
example, objects which support stream interface (e.g., file or UART) provide ``read()``
method which allocates new buffer for read data, but also a ``readinto()`` method
to read data into an existing buffer.
Floating point
~~~~~~~~~~~~~~
Some MicroPython ports allocate floating point numbers on heap. Some other ports
may lack dedicated floating-point coprocessor, and perform arithmetic operations
on them in "software" at considerably lower speed than on integers. Where
performance is important, use integer operations and restrict the use of floating
point to sections of the code where performance is not paramount. For example,
capture ADC readings as integers values to an array in one quick go, and only then
convert them to floating-point numbers for signal processing.
Arrays
~~~~~~
Consider the use of the various types of array classes as an alternative to lists.
The `array` module supports various element types with 8-bit elements supported
by Python's built in `bytes` and `bytearray` classes. These data structures all store
elements in contiguous memory locations. Once again to avoid memory allocation in critical
code these should be pre-allocated and passed as arguments or as bound objects.
When passing slices of objects such as `bytearray` instances, Python creates
a copy which involves allocation of the size proportional to the size of slice.
This can be alleviated using a `memoryview` object. The `memoryview` itself
is allocated on the heap, but is a small, fixed-size object, regardless of the size
of slice it points too. Slicing a `memoryview` creates a new `memoryview`, so this
cannot be done in an interrupt service routine. Further, the slice syntax ``a:b``
causes further allocation by instantiating a ``slice(a, b)`` object.
.. code:: python
ba = bytearray(10000) # big array
func(ba[30:2000]) # a copy is passed, ~2K new allocation
mv = memoryview(ba) # small object is allocated
func(mv[30:2000]) # a pointer to memory is passed
A `memoryview` can only be applied to objects supporting the buffer protocol - this
includes arrays but not lists. Small caveat is that while memoryview object is live,
it also keeps alive the original buffer object. So, a memoryview isn't a universal
panacea. For instance, in the example above, if you are done with 10K buffer and
just need those bytes 30:2000 from it, it may be better to make a slice, and let
the 10K buffer go (be ready for garbage collection), instead of making a
long-living memoryview and keeping 10K blocked for GC.
Nonetheless, `memoryview` is indispensable for advanced preallocated buffer
management. ``readinto()`` method discussed above puts data at the beginning
of buffer and fills in entire buffer. What if you need to put data in the
middle of existing buffer? Just create a memoryview into the needed section
of buffer and pass it to ``readinto()``.
Identifying the slowest section of code
---------------------------------------
This is a process known as profiling and is covered in textbooks and
(for standard Python) supported by various software tools. For the type of
smaller embedded application likely to be running on MicroPython platforms
the slowest function or method can usually be established by judicious use
of the timing ``ticks`` group of functions documented in `time`.
Code execution time can be measured in ms, us, or CPU cycles.
The following enables any function or method to be timed by adding an
``@timed_function`` decorator:
.. code:: python
def timed_function(f, *args, **kwargs):
myname = str(f).split(' ')[1]
def new_func(*args, **kwargs):
t = time.ticks_us()
result = f(*args, **kwargs)
delta = time.ticks_diff(time.ticks_us(), t)
print('Function {} Time = {:6.3f}ms'.format(myname, delta/1000))
return result
return new_func
MicroPython code improvements
-----------------------------
The const() declaration
~~~~~~~~~~~~~~~~~~~~~~~
MicroPython provides a ``const()`` declaration. This works in a similar way
to ``#define`` in C in that when the code is compiled to bytecode the compiler
substitutes the numeric value for the identifier. This avoids a dictionary
lookup at runtime. The argument to ``const()`` may be anything which, at
compile time, evaluates to an integer e.g. ``0x100`` or ``1 << 8``.
.. _Caching:
Caching object references
~~~~~~~~~~~~~~~~~~~~~~~~~~
Where a function or method repeatedly accesses objects performance is improved
by caching the object in a local variable:
.. code:: python
class foo(object):
def __init__(self):
self.ba = bytearray(100)
def bar(self, obj_display):
ba_ref = self.ba
fb = obj_display.framebuffer
# iterative code using these two objects
This avoids the need repeatedly to look up ``self.ba`` and ``obj_display.framebuffer``
in the body of the method ``bar()``.
.. _controlling_gc:
Controlling garbage collection
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
When memory allocation is required, MicroPython attempts to locate an adequately
sized block on the heap. This may fail, usually because the heap is cluttered
with objects which are no longer referenced by code. If a failure occurs, the
process known as garbage collection reclaims the memory used by these redundant
objects and the allocation is then tried again - a process which can take several
milliseconds.
There may be benefits in pre-empting this by periodically issuing `gc.collect()`.
Firstly doing a collection before it is actually required is quicker - typically on the
order of 1ms if done frequently. Secondly you can determine the point in code
where this time is used rather than have a longer delay occur at random points,
possibly in a speed critical section. Finally performing collections regularly
can reduce fragmentation in the heap. Severe fragmentation can lead to
non-recoverable allocation failures.
The Native code emitter
-----------------------
This causes the MicroPython compiler to emit native CPU opcodes rather than
bytecode. It covers the bulk of the MicroPython functionality, so most functions will require
no adaptation (but see below). It is invoked by means of a function decorator:
.. code:: python
@micropython.native
def foo(self, arg):
buf = self.linebuf # Cached object
# code
There are certain limitations in the current implementation of the native code emitter.
* Context managers are not supported (the ``with`` statement).
* Generators are not supported.
* If ``raise`` is used an argument must be supplied.
The trade-off for the improved performance (roughly twice as fast as bytecode) is an
increase in compiled code size.
The Viper code emitter
----------------------
The optimisations discussed above involve standards-compliant Python code. The
Viper code emitter is not fully compliant. It supports special Viper native data types
in pursuit of performance. Integer processing is non-compliant because it uses machine
words: arithmetic on 32 bit hardware is performed modulo 2**32.
Like the Native emitter Viper produces machine instructions but further optimisations
are performed, substantially increasing performance especially for integer arithmetic and
bit manipulations. It is invoked using a decorator:
.. code:: python
@micropython.viper
def foo(self, arg: int) -> int:
# code
As the above fragment illustrates it is beneficial to use Python type hints to assist the Viper optimiser.
Type hints provide information on the data types of arguments and of the return value; these
are a standard Python language feature formally defined here `PEP0484 <https://www.python.org/dev/peps/pep-0484/>`_.
Viper supports its own set of types namely ``int``, ``uint`` (unsigned integer), ``ptr``, ``ptr8``,
``ptr16`` and ``ptr32``. The ``ptrX`` types are discussed below. Currently the ``uint`` type serves
a single purpose: as a type hint for a function return value. If such a function returns ``0xffffffff``
Python will interpret the result as 2**32 -1 rather than as -1.
In addition to the restrictions imposed by the native emitter the following constraints apply:
* Functions may have up to four arguments.
* Default argument values are not permitted.
* Floating point may be used but is not optimised.
Viper provides pointer types to assist the optimiser. These comprise
* ``ptr`` Pointer to an object.
* ``ptr8`` Points to a byte.
* ``ptr16`` Points to a 16 bit half-word.
* ``ptr32`` Points to a 32 bit machine word.
The concept of a pointer may be unfamiliar to Python programmers. It has similarities
to a Python `memoryview` object in that it provides direct access to data stored in memory.
Items are accessed using subscript notation, but slices are not supported: a pointer can return
a single item only. Its purpose is to provide fast random access to data stored in contiguous
memory locations - such as data stored in objects which support the buffer protocol, and
memory-mapped peripheral registers in a microcontroller. It should be noted that programming
using pointers is hazardous: bounds checking is not performed and the compiler does nothing to
prevent buffer overrun errors.
Typical usage is to cache variables:
.. code:: python
@micropython.viper
def foo(self, arg: int) -> int:
buf = ptr8(self.linebuf) # self.linebuf is a bytearray or bytes object
for x in range(20, 30):
bar = buf[x] # Access a data item through the pointer
# code omitted
In this instance the compiler "knows" that ``buf`` is the address of an array of bytes;
it can emit code to rapidly compute the address of ``buf[x]`` at runtime. Where casts are
used to convert objects to Viper native types these should be performed at the start of
the function rather than in critical timing loops as the cast operation can take several
microseconds. The rules for casting are as follows:
* Casting operators are currently: ``int``, ``bool``, ``uint``, ``ptr``, ``ptr8``, ``ptr16`` and ``ptr32``.
* The result of a cast will be a native Viper variable.
* Arguments to a cast can be a Python object or a native Viper variable.
* If argument is a native Viper variable, then cast is a no-op (i.e. costs nothing at runtime)
that just changes the type (e.g. from ``uint`` to ``ptr8``) so that you can then store/load
using this pointer.
* If the argument is a Python object and the cast is ``int`` or ``uint``, then the Python object
must be of integral type and the value of that integral object is returned.
* The argument to a bool cast must be integral type (boolean or integer); when used as a return
type the viper function will return True or False objects.
* If the argument is a Python object and the cast is ``ptr``, ``ptr``, ``ptr16`` or ``ptr32``,
then the Python object must either have the buffer protocol (in which case a pointer to the
start of the buffer is returned) or it must be of integral type (in which case the value of
that integral object is returned).
Writing to a pointer which points to a read-only object will lead to undefined behaviour.
The following example illustrates the use of a ``ptr16`` cast to toggle pin X1 ``n`` times:
.. code:: python
BIT0 = const(1)
@micropython.viper
def toggle_n(n: int):
odr = ptr16(stm.GPIOA + stm.GPIO_ODR)
for _ in range(n):
odr[0] ^= BIT0
A detailed technical description of the three code emitters may be found
on Kickstarter here `Note 1 <https://www.kickstarter.com/projects/214379695/micro-python-python-for-microcontrollers/posts/664832>`_
and here `Note 2 <https://www.kickstarter.com/projects/214379695/micro-python-python-for-microcontrollers/posts/665145>`_
Accessing hardware directly
---------------------------
.. note::
Code examples in this section are given for the Pyboard. The techniques
described however may be applied to other MicroPython ports too.
This comes into the category of more advanced programming and involves some knowledge
of the target MCU. Consider the example of toggling an output pin on the Pyboard. The
standard approach would be to write
.. code:: python
mypin.value(mypin.value() ^ 1) # mypin was instantiated as an output pin
This involves the overhead of two calls to the :class:`~machine.Pin` instance's :meth:`~machine.Pin.value()`
method. This overhead can be eliminated by performing a read/write to the relevant bit
of the chip's GPIO port output data register (odr). To facilitate this the ``stm``
module provides a set of constants providing the addresses of the relevant registers.
A fast toggle of pin ``P4`` (CPU pin ``A14``) - corresponding to the green LED -
can be performed as follows:
.. code:: python
import machine
import stm
BIT14 = const(1 << 14)
machine.mem16[stm.GPIOA + stm.GPIO_ODR] ^= BIT14