Skip to content
项目
群组
代码片段
帮助
当前项目
正在载入...
登录 / 注册
切换导航面板
P
pytensor
项目
项目
详情
活动
周期分析
仓库
仓库
文件
提交
分支
标签
贡献者
图表
比较
统计图
议题
0
议题
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
CI / CD
CI / CD
流水线
作业
日程
统计图
Wiki
Wiki
代码片段
代码片段
成员
成员
折叠边栏
关闭边栏
活动
图像
聊天
创建新问题
作业
提交
问题看板
Open sidebar
testgroup
pytensor
Commits
91303337
提交
91303337
authored
8月 10, 2022
作者:
Brandon T. Willard
提交者:
Brandon T. Willard
8月 12, 2022
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Fix type hints and docstrings in aesara.link.c.cmodule
上级
d09a80d0
隐藏空白字符变更
内嵌
并排
正在显示
1 个修改的文件
包含
30 行增加
和
26 行删除
+30
-26
cmodule.py
aesara/link/c/cmodule.py
+30
-26
没有找到文件。
aesara/link/c/cmodule.py
浏览文件 @
91303337
...
@@ -19,7 +19,7 @@ import textwrap
...
@@ -19,7 +19,7 @@ import textwrap
import
time
import
time
import
warnings
import
warnings
from
io
import
BytesIO
,
StringIO
from
io
import
BytesIO
,
StringIO
from
typing
import
Callable
,
Dict
,
List
,
Optional
,
Set
,
Tuple
,
cast
from
typing
import
TYPE_CHECKING
,
Callable
,
Dict
,
List
,
Optional
,
Set
,
Tuple
,
cast
import
numpy
as
np
import
numpy
as
np
from
setuptools._distutils.sysconfig
import
(
from
setuptools._distutils.sysconfig
import
(
...
@@ -46,6 +46,10 @@ from aesara.utils import (
...
@@ -46,6 +46,10 @@ from aesara.utils import (
)
)
if
TYPE_CHECKING
:
from
aesara.link.c.basic
import
CLinker
class
StdLibDirsAndLibsType
(
Protocol
):
class
StdLibDirsAndLibsType
(
Protocol
):
data
:
Optional
[
Tuple
[
List
[
str
],
...
]]
data
:
Optional
[
Tuple
[
List
[
str
],
...
]]
__call__
:
Callable
[[],
Optional
[
Tuple
[
List
[
str
],
...
]]]
__call__
:
Callable
[[],
Optional
[
Tuple
[
List
[
str
],
...
]]]
...
@@ -616,38 +620,38 @@ class ModuleCache:
...
@@ -616,38 +620,38 @@ class ModuleCache:
can import.
can import.
The cache contains one directory for each module, containing:
The cache contains one directory for each module, containing:
- the dynamic library file itself (
.so/.pyd
),
- the dynamic library file itself (
e.g. ``.so/.pyd``
),
- an empty
__init__.py
file, so Python can import it,
- an empty
``__init__.py``
file, so Python can import it,
- a file containing the source code for the module (
mod.cpp/mod.cu
),
- a file containing the source code for the module (
e.g. ``mod.cpp/mod.cu``
),
- a
key.pkl
file, containing a KeyData object with all the keys
- a
``key.pkl``
file, containing a KeyData object with all the keys
associated with that module,
associated with that module,
- possibly a
delete.me
file, meaning this directory has been marked
- possibly a
``delete.me``
file, meaning this directory has been marked
for deletion.
for deletion.
Keys should be tuples of length
2: (version, rest)
. The
Keys should be tuples of length
two: ``(version, rest)``
. The
``rest``
can be anything hashable and picklable, that uniquely
rest
can be anything hashable and picklable, that uniquely
identifies the computation in the module. The key is returned by
identifies the computation in the module. The key is returned by
`
`CLinker.cmodule_key_`
`.
`
CLinker.cmodule_key_
`.
The ``version``
should be a hierarchy of tuples of integers.
The ``version``
value should be a hierarchy of tuples of integers. If the
If the ``version`` is either 0 or (), then the key is unversioned, and its
``version`` value is either ``0`` or ``()``, then the key is unversioned,
corresponding module will be marked for deletion in an atexit() handler.
and its corresponding module will be marked for deletion in an `atexit`
If the ``version`` is neither 0 nor (), then the module will be kept i
n the
handler. If the ``version`` value is neither ``0`` nor ``()``, the
n the
cache between processes.
module will be kept in the
cache between processes.
An unversioned module is not always deleted by the process that
An unversioned module is not always deleted by the process that
creates it. Deleting such modules may not work on NFS filesystems
creates it. Deleting such modules may not work on NFS filesystems
because the tmpdir in which the library resides is in use until the
because the tmpdir in which the library resides is in use until the
end of the process' lifetime. In this case, unversioned modules
end of the process' lifetime. In this case, unversioned modules
are left in their t
mpdirs without corresponding .pkl files. These
are left in their t
emporary directories without corresponding ``.pkl``
modules and their directories are erased by subsequent processes'
files. These modules and their directories are erased by subsequent
refresh()
functions.
processes' `ModuleCache.refresh`
functions.
Two different keys are mapped to the same module when all conditions below
Two different keys are mapped to the same module when all conditions below
are met:
are met:
- They have the same version.
- They have the same version.
- They share the same compilation options in their ``rest`` part (see
- They share the same compilation options in their ``rest`` part (see
`
`CLinker.cmodule_key_`
` for how this part is built).
`
CLinker.cmodule_key_
` for how this part is built).
- They share the same C code.
- They share the same C code.
These three elements uniquely identify a module, and are summarized
These three elements uniquely identify a module, and are summarized
in a single "module hash".
in a single "module hash".
...
@@ -655,12 +659,12 @@ class ModuleCache:
...
@@ -655,12 +659,12 @@ class ModuleCache:
Parameters
Parameters
----------
----------
check_for_broken_eq
check_for_broken_eq
A bad
__eq__
implementation can break this cache mechanism.
A bad
`object.__eq__`
implementation can break this cache mechanism.
This option turns on a not-too-expensive sanity check every
This option turns on a not-too-expensive sanity check every
time a new key is added to the cache.
time a new key is added to the cache.
do_refresh : bool
do_refresh : bool
If
True, then the ``refresh`
` method will be called
If
``True``, then the `ModuleCache.refresh
` method will be called
in the constructor.
in the constructor.
"""
"""
...
@@ -677,7 +681,7 @@ class ModuleCache:
...
@@ -677,7 +681,7 @@ class ModuleCache:
"""
"""
entry_from_key
:
Dict
=
{}
entry_from_key
:
Dict
=
{}
"""
"""
Maps keys to the filename of a
.so/.pyd
.
Maps keys to the filename of a
``.so/.pyd``
.
"""
"""
similar_keys
:
Dict
=
{}
similar_keys
:
Dict
=
{}
...
@@ -687,18 +691,18 @@ class ModuleCache:
...
@@ -687,18 +691,18 @@ class ModuleCache:
"""
"""
module_hash_to_key_data
:
Dict
=
{}
module_hash_to_key_data
:
Dict
=
{}
"""
"""
Maps a module hash to its corresponding
KeyData
object.
Maps a module hash to its corresponding
`KeyData`
object.
"""
"""
stats
:
List
=
[]
stats
:
List
=
[]
"""
"""
A list with counters for the number of hits, loads, compiles issued by
A list with counters for the number of hits, loads, compiles issued by
module_from_key()
.
`ModuleCache.module_from_key`
.
"""
"""
loaded_key_pkl
:
Set
=
set
()
loaded_key_pkl
:
Set
=
set
()
"""
"""
Set of all
key.pkl
files that have been loaded.
Set of all
``key.pkl``
files that have been loaded.
"""
"""
...
@@ -1164,7 +1168,7 @@ class ModuleCache:
...
@@ -1164,7 +1168,7 @@ class ModuleCache:
self
.
_update_mappings
(
key
,
key_data
,
module
.
__file__
,
not
key_broken
)
self
.
_update_mappings
(
key
,
key_data
,
module
.
__file__
,
not
key_broken
)
return
key_data
return
key_data
def
module_from_key
(
self
,
key
,
lnk
=
None
):
def
module_from_key
(
self
,
key
,
lnk
:
"CLinker"
):
"""
"""
Return a module from the cache, compiling it if necessary.
Return a module from the cache, compiling it if necessary.
...
@@ -1319,7 +1323,7 @@ class ModuleCache:
...
@@ -1319,7 +1323,7 @@ class ModuleCache:
age_thresh_del
=
config
.
cmodule__age_thresh_use
+
60
*
60
*
24
*
7
age_thresh_del
=
config
.
cmodule__age_thresh_use
+
60
*
60
*
24
*
7
age_thresh_del_unversioned
=
60
*
60
*
24
*
7
# 7 days
age_thresh_del_unversioned
=
60
*
60
*
24
*
7
# 7 days
"""
"""
The default age threshold for `clear_old` (in seconds).
The default age threshold for `
ModuleCache.
clear_old` (in seconds).
"""
"""
...
...
编写
预览
Markdown
格式
0%
重试
或
添加新文件
添加附件
取消
您添加了
0
人
到此讨论。请谨慎行事。
请先完成此评论的编辑!
取消
请
注册
或者
登录
后发表评论