forked from NCAS-CMS/cf-python
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathperformance.html
More file actions
390 lines (353 loc) · 24.4 KB
/
performance.html
File metadata and controls
390 lines (353 loc) · 24.4 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta charset="utf-8" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
<title>Performance — Documentation</title>
<link rel="stylesheet" href="_static/alabaster.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="_static/graphviz.css" />
<link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
<link rel="stylesheet" type="text/css" href="_static/customise-alabaster.css" />
<script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<script type="text/javascript" src="_static/language_data.js"></script>
<script type="text/javascript" src="_static/clipboard.min.js"></script>
<script type="text/javascript" src="_static/copybutton.js"></script>
<script type="text/javascript" src="_static/toggleprompt.js"></script>
<script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Releases" href="releases.html" />
<link rel="prev" title="Aggregation rules" href="aggregation_rules.html" />
<link rel="stylesheet" href="_static/custom.css" type="text/css" />
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
</head><body>
<div class="document">
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h1 class="logo"><a href="index.html">cf 3.13.0</a></h1>
<p class="blurb">A CF-compliant earth science data analysis library</p>
<p>
<iframe src="https://ghbtns.com/github-btn.html?user=NCAS-CMS&repo=cf-python&type=star&count=true&size=large&v=2"
allowtransparency="true" frameborder="0" scrolling="0" width="200px" height="35px"></iframe>
</p>
<div id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" />
<input type="submit" value="Go" />
</form>
</div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="introduction.html"><strong>Introduction</strong></a></li>
<li class="toctree-l1"><a class="reference internal" href="cf_data_model.html"><strong>CF data model</strong></a></li>
<li class="toctree-l1"><a class="reference internal" href="installation.html"><strong>Installation</strong></a></li>
<li class="toctree-l1"><a class="reference internal" href="contributing.html"><strong>Contributing</strong></a></li>
<li class="toctree-l1"><a class="reference internal" href="tutorial.html"><strong>Tutorial</strong></a></li>
<li class="toctree-l1"><a class="reference internal" href="analysis.html"><strong>Analysis</strong></a></li>
<li class="toctree-l1"><a class="reference internal" href="api_reference.html"><strong>API reference</strong></a></li>
<li class="toctree-l1"><a class="reference internal" href="aggregation_rules.html"><strong>Aggregation rules</strong></a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#"><strong>Performance</strong></a></li>
<li class="toctree-l1"><a class="reference internal" href="releases.html"><strong>Releases</strong></a></li>
<li class="toctree-l1"><a class="reference internal" href="Changelog.html"><strong>Change log</strong></a></li>
</ul>
<div class="relations">
<h3>Related Topics</h3>
<ul>
<li><a href="index.html">Documentation overview</a><ul>
<li>Previous: <a href="aggregation_rules.html" title="previous chapter"><strong>Aggregation rules</strong></a></li>
<li>Next: <a href="releases.html" title="next chapter"><strong>Releases</strong></a></li>
</ul></li>
</ul>
</div>
<br>
cf development has been supported by
the <a href="https://erc.europa.eu/">ERC</a>
through <a href="https://cordis.europa.eu/project/id/247220">Seachange</a>
and
<a href="https://cordis.europa.eu/project/id/786427">Couplet</a>; by
the <a href="https://ec.europa.eu/programmes/horizon2020/">EC Horizon
2020 programme</a> through
<a href="https://cordis.europa.eu/project/id/824084">IS-ENES3</a>;
by <a href="https://nerc.ukri.org/">NERC</a> through
<a href="https://gtr.ukri.org/project/0D95A6DB-0B95-48F7-8A8B-7B9A47DEA117">UKFAFMIP</a>;
and by <a href="https://ncas.ac.uk/">NCAS</a>.
<br>
<br>
<img src="_templates/logo_EC.png" height="40">
<img src="_templates/logo_ERC.png" height="40">
<img src="_templates/logo_NERC.png" height="40">
<br>
<img src="_templates/logo_NCAS.png" height="40">
</div>
</div>
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<section id="performance">
<h1><strong>Performance</strong><a class="headerlink" href="#performance" title="Permalink to this headline">¶</a></h1>
<hr class="docutils" />
<p>Version 3.13.0 for version 1.9 of the CF conventions.</p>
<div class="contents local topic" id="contents">
<ul class="simple">
<li><p><a class="reference internal" href="#lazy-operations" id="id3"><strong>Lazy operations</strong></a></p></li>
<li><p><a class="reference internal" href="#regridding" id="id4"><strong>Regridding</strong></a></p></li>
<li><p><a class="reference internal" href="#large-amounts-of-massive-arrays-lama" id="id5"><strong>Large Amounts of Massive Arrays (LAMA)</strong></a></p>
<ul>
<li><p><a class="reference internal" href="#reading-from-files" id="id6">Reading from files</a></p></li>
<li><p><a class="reference internal" href="#copying" id="id7">Copying</a></p></li>
<li><p><a class="reference internal" href="#aggregation" id="id8">Aggregation</a></p></li>
<li><p><a class="reference internal" href="#subspacing" id="id9">Subspacing</a></p></li>
<li><p><a class="reference internal" href="#speed-and-memory-management" id="id10">Speed and memory management</a></p></li>
<li><p><a class="reference internal" href="#temporary-files" id="id11">Temporary files</a></p></li>
<li><p><a class="reference internal" href="#partitioning" id="id12">Partitioning</a></p></li>
</ul>
</li>
<li><p><a class="reference internal" href="#parallelization" id="id13"><strong>Parallelization</strong></a></p></li>
</ul>
</div>
<section id="lazy-operations">
<h2><a class="toc-backref" href="#id3"><strong>Lazy operations</strong></a><a class="headerlink" href="#lazy-operations" title="Permalink to this headline">¶</a></h2>
<p>Many data operations of field and metadata constructs are lazy
i.e. the operation is actually performed until the result is needed,
or the underlying data are shared with a <a class="reference external" href="https://en.wikipedia.org/wiki/Copy-on-write">copy-on-write</a> technique, or <a class="reference external" href="https://en.wikipedia.org/wiki/Lazy_loading">lazy
loading</a> delays reading
data from disk until it is needed. These are:</p>
<ul class="simple">
<li><p>Reading datasets from disk</p></li>
<li><p>Subspacing (in index and metadata space)</p></li>
<li><p>Axis manipulations (such as rearranging the axis order of the data)</p></li>
<li><p>Copying</p></li>
<li><p>Field construct aggregation (and data concatenation in general)</p></li>
<li><p>Changing the units</p></li>
</ul>
<p>Operations that involve changing the data values (apart from changing
the units) are not lazy; such arithmetic, trigonometrical, rounding,
collapse, etc. functions.</p>
<p>All of these lazy techniques make use of <a class="reference internal" href="#lama"><span class="std std-ref">LAMA (Large Amounts
of Massive Arrays)</span></a> functionality.</p>
</section>
<hr class="docutils" />
<section id="regridding">
<h2><a class="toc-backref" href="#id4"><strong>Regridding</strong></a><a class="headerlink" href="#regridding" title="Permalink to this headline">¶</a></h2>
<p>When regridding multiple field constructs with their
<a class="reference internal" href="method/cf.Field.regrids.html#cf.Field.regrids" title="cf.Field.regrids"><code class="xref py py-obj docutils literal notranslate"><span class="pre">cf.Field.regrids</span></code></a> or <a class="reference internal" href="method/cf.Field.regridc.html#cf.Field.regridc" title="cf.Field.regridc"><code class="xref py py-obj docutils literal notranslate"><span class="pre">cf.Field.regridc</span></code></a> methods, the regridding
weights are calculated for each field’s operation, and these
calculations are often the main expense of the operation, sometimes
close to 100% of the cost.</p>
<p>This can be avoided for cases when all of the fields are being
regridded to the same destination grid, and all share a common source
grid. In such cases, the regridding operator that defines the grids
and the weights can be calculated once and then used to regrid each
field construct, giving potentially very large performance
improvements.</p>
<div class="literal-block-wrapper docutils container" id="id2">
<div class="code-block-caption"><span class="caption-text"><em>Regrid every field construct in the field list ‘fl’ to a
common destination grid defined by the field construct
‘dst’.</em></span><a class="headerlink" href="#id2" title="Permalink to this code">¶</a></div>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">operator</span> <span class="o">=</span> <span class="n">fl</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">regrids</span><span class="p">(</span><span class="n">dst</span><span class="p">,</span> <span class="n">method</span><span class="o">=</span><span class="s1">'conservative'</span><span class="p">,</span>
<span class="gp">... </span> <span class="n">return_operator</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">fl_regridded</span> <span class="o">=</span> <span class="n">cf</span><span class="o">.</span><span class="n">FieldList</span><span class="p">(</span><span class="n">f</span><span class="o">.</span><span class="n">regrids</span><span class="p">(</span><span class="n">operator</span><span class="p">)</span> <span class="k">for</span> <span class="n">f</span> <span class="ow">in</span> <span class="n">fl</span><span class="p">)</span>
</pre></div>
</div>
</div>
<hr class="docutils" />
</section>
<section id="large-amounts-of-massive-arrays-lama">
<span id="lama"></span><h2><a class="toc-backref" href="#id5"><strong>Large Amounts of Massive Arrays (LAMA)</strong></a><a class="headerlink" href="#large-amounts-of-massive-arrays-lama" title="Permalink to this headline">¶</a></h2>
<p>Data are stored and manipulated in a very memory efficient manner such
that large numbers of constructs may co-exist and be manipulated
regardless of the size of their data arrays. The limiting factor on
array size is not the amount of available physical memory, but the
amount of disk space available, which is generally far greater.</p>
<p>The basic functionality is:</p>
<ul>
<li><p><strong>Arrays larger than the available physical memory may be created.</strong></p>
<p>Arrays larger than a preset number of bytes are partitioned into
smaller sub-arrays which are not kept in memory but stored on disk,
either as part of the original file they were read in from or as
newly created temporary files, whichever is appropriate. Therefore
data arrays larger than a preset size need never wholly exist in
memory.</p>
</li>
<li><p><strong>Large numbers of arrays which are in total larger than the
available physical memory may co-exist.</strong></p>
<p>Large numbers of data arrays which are collectively larger than the
available physical memory may co-exist, regardless of whether any
individual array is in memory or stored on disk.</p>
</li>
<li><p><strong>Arrays larger than the available physical memory may be operated
on.</strong></p>
<p>Array operations (such as subspacing, assignment, arithmetic,
comparison, collapses, etc.) are carried out on a
partition-by-partition basis. When an operation traverses the
partitions, if a partition is stored on disk then it is returned to
disk before processing the next partition.</p>
</li>
<li><p><strong>The memory management does not need to be known at the API
level.</strong></p>
<p>As the array’s metadata (such as size, shape, data-type, number of
dimensions, etc.) always reflect the data array in its entirety,
regardless of how the array is partitioned and whether or not its
partitions are in memory, constructs containing data arrays may be
used in the API as if they were normal, in-memory objects (like
<a class="reference external" href="https://numpy.org/doc/stable/reference/index.html#module-numpy" title="(in NumPy v1.23)"><code class="xref py py-obj docutils literal notranslate"><span class="pre">numpy</span></code></a> arrays). Partitioning does carry a performance overhead, but
this may be minimised for particular applications or hardware
configurations.</p>
</li>
</ul>
<section id="reading-from-files">
<span id="lama-reading"></span><h3><a class="toc-backref" href="#id6">Reading from files</a><a class="headerlink" href="#reading-from-files" title="Permalink to this headline">¶</a></h3>
<p>When a field construct is read from a file, the data array is not
realized in memory, however large or small it may be. Instead each
partition refers to part of the original file on disk. Therefore
reading even very large field constructs is initially very fast and
uses up only a very small amount of memory.</p>
</section>
<section id="copying">
<span id="lama-copying"></span><h3><a class="toc-backref" href="#id7">Copying</a><a class="headerlink" href="#copying" title="Permalink to this headline">¶</a></h3>
<p>When a field construct is deep copied with its <a class="reference internal" href="method/cf.Field.copy.html#cf.Field.copy" title="cf.Field.copy"><code class="xref py py-obj docutils literal notranslate"><span class="pre">copy</span></code></a> method or
the Python <a class="reference external" href="https://docs.python.org/3/library/copy.html#copy.deepcopy" title="(in Python v3.10)"><code class="xref py py-obj docutils literal notranslate"><span class="pre">copy.deepcopy</span></code></a> function, the partitions of its data array
are transferred to the new field construct as object identities and
are <em>not</em> deep copied. Therefore copying even very large field
constructs is initially very fast and uses up only a very small amount
of memory.</p>
<p>The independence of the copied field construct is preserved, however,
since each partition that is stored in memory (as opposed to on disk)
is deep copied if and when the data in the new field construct is
actually accessed, and then only if the partition’s data still exists
in the original (or any other) field construct.</p>
</section>
<section id="aggregation">
<h3><a class="toc-backref" href="#id8">Aggregation</a><a class="headerlink" href="#aggregation" title="Permalink to this headline">¶</a></h3>
<p>When two field constructs are aggregated to form one, larger field
construct there is no need for either field construct’s data array
partitions to be accessed, and therefore brought into memory if they
were stored on disk. The resulting field construct recombines the two
field constructs’ array partitions as object identities into the new
larger array. Thereby creating an aggregated field construct that uses
up only a very small amount of extra memory.</p>
<p>The independence of the new field construct is preserved, however,
since each partition that is stored in memory (as opposed to on disk)
is deep copied when the data in the new field construct is actually
accessed, and then only if the partition’s data still exists in its
the original (or any other) field construct.</p>
</section>
<section id="subspacing">
<span id="lama-subspacing"></span><h3><a class="toc-backref" href="#id9">Subspacing</a><a class="headerlink" href="#subspacing" title="Permalink to this headline">¶</a></h3>
<p>When a new field construct is created by <a class="reference external" href="https://ncas-cms.github.io/cfdm/tutorial.html#subspacing" title="(in Python cfdm package v1.9)"><span class="xref std std-ref">subspacing</span></a> a field construct, the new field construct is actually a
<a class="reference internal" href="#lama-copying"><span class="std std-ref">LAMA deep copy</span></a> of the original but with
additional instructions on each of its data partitions to only use the
part specified by the subspace indices.</p>
<p>As with copying, creating subspaced field constructs is initially very
fast and uses up only a very small amount of memory, with the added
advantage that a deep copy of only the requested parts of the data
array needs to be carried out at the time of data access, and then
only if the partition’s data still exists in the original field
construct.</p>
<p>When subspacing a field construct that has previously been subspacing
but has not yet had its data accessed, the new subspace merely updates
the instructions on which parts of the array partitions to use. For
example:</p>
<div class="doctest highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">f</span><span class="o">.</span><span class="n">shape</span> <span class="o">=</span> <span class="p">(</span><span class="mi">12</span><span class="p">,</span> <span class="mi">73</span><span class="p">,</span> <span class="mi">96</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">g</span> <span class="o">=</span> <span class="n">f</span><span class="p">[::</span><span class="o">-</span><span class="mi">2</span><span class="p">,</span> <span class="o">...</span><span class="p">]</span>
<span class="gp">>>> </span><span class="n">h</span> <span class="o">=</span> <span class="n">g</span><span class="p">[</span><span class="mi">2</span><span class="p">:</span><span class="mi">5</span><span class="p">,</span> <span class="o">...</span><span class="p">]</span>
</pre></div>
</div>
<p>is equivalent to</p>
<div class="doctest highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">h</span> <span class="o">=</span> <span class="n">f</span><span class="p">[</span><span class="mi">7</span><span class="p">:</span><span class="mi">2</span><span class="p">:</span><span class="o">-</span><span class="mi">2</span><span class="p">,</span> <span class="o">...</span><span class="p">]</span>
</pre></div>
</div>
<p>and if all of the partitions of field construct <code class="docutils literal notranslate"><span class="pre">f</span></code> are stored on
disk then in both cases so are all of the partitions of field
construct <code class="docutils literal notranslate"><span class="pre">h</span></code> and no data has been read from disk.</p>
</section>
<section id="speed-and-memory-management">
<h3><a class="toc-backref" href="#id10">Speed and memory management</a><a class="headerlink" href="#speed-and-memory-management" title="Permalink to this headline">¶</a></h3>
<p>The creation of temporary files for array partitions of large arrays
and the reading of data from files on disk can create significant
speed overheads (for example, recent tests show that writing a 100
megabyte array to disk can take O(1) seconds), so it may be desirable
to configure the maximum size of array which is kept in memory, and
therefore has fast access.</p>
<p>The data array memory management is configurable as follows:</p>
<ul class="simple">
<li><p>Data arrays larger than a given size are partitioned into
sub-arrays, each of which is smaller than the chunk size. By default
this size is set to 1% of the total physical memory and is found and
set with the <a class="reference internal" href="function/cf.chunksize.html#cf.chunksize" title="cf.chunksize"><code class="xref py py-obj docutils literal notranslate"><span class="pre">cf.chunksize</span></code></a> function.</p></li>
<li><p>In-memory sub-arrays may be written to temporary files on disk when
the amount of available physical memory falls below a specified
amount. By default this amount is 10% of the total physical memory
and is found and set with the <code class="xref py py-obj docutils literal notranslate"><span class="pre">cf.MINNCFCM</span></code> function.</p></li>
</ul>
</section>
<section id="temporary-files">
<span id="lama-temporary-files"></span><h3><a class="toc-backref" href="#id11">Temporary files</a><a class="headerlink" href="#temporary-files" title="Permalink to this headline">¶</a></h3>
<p>The directory in which temporary files is found and set with the
<a class="reference internal" href="function/cf.tempdir.html#cf.tempdir" title="cf.tempdir"><code class="xref py py-obj docutils literal notranslate"><span class="pre">cf.tempdir</span></code></a> function:</p>
<div class="doctest highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">cf</span><span class="o">.</span><span class="n">tempdir</span><span class="p">()</span>
<span class="go">'/tmp'</span>
<span class="gp">>>> </span><span class="n">cf</span><span class="o">.</span><span class="n">tempdir</span><span class="p">(</span><span class="s1">'/home/me/tmp'</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">cf</span><span class="o">.</span><span class="n">tempdir</span><span class="p">()</span>
<span class="go">'/home/me/tmp'</span>
</pre></div>
</div>
<p>The removal of temporary files which are no longer required works in
the same way as python’s automatic garbage collector. When a
partition’s data is stored in a temporary file, that file will only
exist for as long as there are partitions referring to it. When no
partitions require the file it will be deleted automatically.</p>
<p>When python exits normally, all temporary files are always deleted.</p>
<p>Changing the temporary file directory does not prevent temporary files
in the original directory from being garbage collected.</p>
</section>
<section id="partitioning">
<h3><a class="toc-backref" href="#id12">Partitioning</a><a class="headerlink" href="#partitioning" title="Permalink to this headline">¶</a></h3>
<p>Data partitioning preserves as much as is possible the faster varying
(inner) dimensions’ sizes in each of the sub-arrays.</p>
<p><strong>Examples</strong></p>
<ul class="simple">
<li><p>If an array with shape (2, 4, 6) is partitioned into 2 partitions
then both sub-arrays will have shape (1, 4, 6).</p></li>
<li><p>If the same array is partitioned into 4 partitions then all four
sub-arrays will have shape (1, 2, 6).</p></li>
<li><p>If the same array is partitioned into 8 partitions then all eight
sub-arrays will have shape (1, 2, 3).</p></li>
<li><p>If the same array is partitioned into 48 partitions then all forty
eight sub-arrays will have shape (1, 1, 1).</p></li>
</ul>
<hr class="docutils" />
</section>
</section>
<section id="parallelization">
<span id="id1"></span><h2><a class="toc-backref" href="#id13"><strong>Parallelization</strong></a><a class="headerlink" href="#parallelization" title="Permalink to this headline">¶</a></h2>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>The experimental ability to run cf scripts in parallel using
mpirun was removed at version 3.9.0. This functionality will
be restored in version 3.14.0 in a robust fashion when the
move to using <code class="xref py py-obj docutils literal notranslate"><span class="pre">dask</span></code>
(<a class="reference external" href="https://github.com/NCAS-CMS/cf-python/issues/182">https://github.com/NCAS-CMS/cf-python/issues/182</a>) is
completed.</p>
</div>
</section>
</section>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
©2022, NCAS | Page built on 2022-07-11.
|
Powered by <a href="http://sphinx-doc.org/">Sphinx 2.3.1</a>
& <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.12</a>
</div>
</body>
</html>