2.5.0

PyKX 2.5.0 has been released. Full release notes can be found here.

Highlights:

• table.xbar , table.window_join , table.replace
• Added as_arrow keyword to the .pd() method to use PyArrow backed data types rather than NumPy.
• Other items of note:
• PyKX can now be installed to locations with spaces in the file path.
• Updated libq to 4.0 2024.05.07 and 4.1 to 2024.04.29 for all supported OS's.
• IPC queries can now pass PyKX Functions like objects as the first query parameter.
• k4.lic licenses can now be installed using the interactive license helper.
• To ease license updates, If PyKX fails to start due to a license error it will attempt to replace it's license from KDB_LICENSE_B64 or KDB_K4LICENSE_B64 if you have one set.

The full list including more fixes and improvements is available here.

Examples:

table.window_join

>>> trades = kx.Table(data={
...     'sym': ['ibm', 'ibm', 'ibm'],
...     'time': kx.q('10:01:01 10:01:04 10:01:08'),
...     'price': [100, 101, 105]})
>>> quotes = kx.Table(data={
...     'sym': 'ibm',
...     'time': kx.q('10:01:01+til 9'),
...     'ask': [101, 103, 103, 104, 104, 107, 108, 107, 108],
...     'bid': [98, 99, 102, 103, 103, 104, 106, 106, 107, 108]})
>>> windows = kx.q('{-2 1+\:x}', trades['time'])
    >>> trades.window_join(quotes,
    ...                    windows,
    ...                    ['sym', 'time'],
    ...                    {'ask_minus_bid': [lambda x, y: x - y, 'ask', 'bid'],
    ...                     'ask_max': [lambda x: max(x), 'ask']})
pykx.Table(pykx.q('
    sym time     price ask_minus_bid ask_max
    ----------------------------------------
    ibm 10:01:01 100   3 4           103
    ibm 10:01:04 101   4 1 1 1       104
    ibm 10:01:08 105   3 2 1 1       108
'))

table.xbar

>>> kx.random.seed(42)
>>> tab = kx.Table(data = {
...     'x': kx.random.random(N, 100.0),
...     'y': kx.random.random(N, 10.0)})
>>> tab
pykx.Table(pykx.q('
x        y
-----------------
77.42128 8.200469
70.49724 9.857311
52.12126 4.629496
99.96985 8.518719
1.196618 9.572477
'))
>>> tab.xbar('x', 10)
pykx.Table(pykx.q('
x  y
-----------
70 8.200469
70 9.857311
50 4.629496
90 8.518719
0  9.572477
'))

table.replace

>>> tab = kx.q('([] a:2 2 3; b:4 2 6; c:(1b;0b;1b); d:(`a;`b;`c); e:(1;2;`a))')
>>> tab
pykx.Table(pykx.q('
a b c d e
----------
2 4 1 a 1
2 2 0 b 2
3 6 1 c `a
'))
>>> tab.replace(2, "test")
pykx.Table(pykx.q('
a     b     c d e
---------------------
`test 4     1 a 1
`test `test 0 b `test
3     6     1 c `a
'))

2.4.0

Full details on the release can be found here.

Additions:

• Support for q/kdb+ 4.1 documentation here added as an opt-in capability, this functionality is enabled through setting PYKX_4_1_ENABLED environment variable.

>>> import os
>>> os.environ['PYKX_4_1_ENABLED'] = 'True'
>>> import pykx as kx
>>> kx.q.z.K
pykx.FloatAtom(pykx.q('4.1'))

• Added support for Python 3.12.
  • Support for PyArrow in this python version is currently in Beta.
• Added conversion of NumPy arrays of type datetime64[s], datetime64[ms], datetime64[us] to kx.TimestampVector
• Added Table.sort_values(), Table.nsmallest() and Table.nlargest() to the Pandas like API for sorting tables.
• Table.rename() now supports non-numerical index columns and improved the quality of errors thrown.
• Added the reconnection_attempts key word argument to SyncQConnection, SecureQConnection, and AsyncQConnection IPC classes. This argument allows IPC connection to be automatically re-established when it is lost and a server has reinitialized.

>>> import pykx as kx
>>> conn = kx.SyncQConnection(port = 5050, reconnection_attempts=4)
>>> conn('1+1')    # Following this call the server on port 5050 was closed for 2 seconds
pykx.LongVector(pykx.q('2'))
>>> conn('1+2')
WARNING: Connection lost attempting to reconnect.
Failed to reconnect, trying again in 0.5 seconds.
Failed to reconnect, trying again in 1.0 seconds.
Connection successfully reestablished.
pykx.LongAtom(pykx.q('3'))

• Added --reconnection_attempts option to Jupyter %%q magic making use of the above IPC logic changes.
• Addition of environment variable/configuration value PYKX_QDEBUG which allows debugging backtrace to be displayed for all calls into q instead of requiring a user to specify debugging is enabled per-call. This additionally works for remote IPC calls and utilisation of Jupyter magic commands.

>>> import os
>>> os.environ['PYKX_QDEBUG'] = 'True'
>>> import pykx as kx
>>> kx.q('{x+1}', 'e')
backtrace:
  [2]  {x+1}
         ^
  [1]  (.Q.trp)

  [0]  {[pykxquery] .Q.trp[value; pykxquery; {if[y~();:(::)];2@"backtrace:
                    ^
",.Q.sbt y;'x}]}
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/anaconda3/lib/python3.8/site-packages/pykx/embedded_q.py", line 230, in __call__
    return factory(result, False)
  File "pykx/_wrappers.pyx", line 493, in pykx._wrappers._factory
  File "pykx/_wrappers.pyx", line 486, in pykx._wrappers.factory
pykx.exceptions.QError: type

Fixes and Improvements:

• Resolved segfaults on Windows when PyKX calls Python functions under q.

>>> import pykx as kx
>>> kx.q('{[f;x] f  x}', sum, kx.q('4 4#til 16'))
pykx.LongVector(pykx.q('24 28 32 36'))

• Updated kdb Insights Core libraries to 4.0.8, see here for more information.
• Updated libq 4.0 version to 2024.03.04 for all supported OS’s.
• Fix issue where use of valid C backed q code APIs could result in segmentation faults when called.

>>> import pykx as kx
>>> isf = kx.q('.pykx.util.isf')
>>> isf
pykx.Foreign(pykx.q('code'))
>>> isf(True)
pykx.BooleanAtom(pykx.q('0b'))

• Each call to the PyKX query API interned 3 new unique symbols. This has now been removed.

Beta Features

• Addition of Compress and Encrypt classes to allow users to set global configuration and for usage within Database partition persistence.

Standalone

>>> import pykx as kx
>>> compress = kx.Compress(algo=kx.CompressionAlgorithm.gzip, level=8)
>>> kx.q.z.zd
pykx.Identity(pykx.q('::'))
>>> compress.global_init()
pykx.LongVector(pykx.q('17 2 8'))
>>> encrypt = kx.Encrypt(path='/path/to/the.key', password='PassWord')
>>> encrypt.load_key()

Database

>>> import pykx as kx
>>> compress = kx.Compress(algo=kx.CompressionAlgorithm.lz4hc, level=10)
>>> db = kx.DB(path='/tmp/db')
>>> db.create(kx.q('([]10?1f;10?1f)', 'tab', kx.q('2020.03m'), compress=compress)
>>> kx.q('-21!`:/tmp/db/2020.03/tab/x')
pykx.Dictionary(pykx.q('
compressedLength  | 140
uncompressedLength| 96
algorithm         | 4i
logicalBlockSize  | 17i
zipLevel          | 10i
'))

2.3.2

Merge pull request #21 from KxSystems/pykx-2.3.2

PyKX 2.3.2 release update

2.3.1

Release 2.3.1

2.2.0

Release 2.2