 |
RedisX v1.0
A simple, light-weight Redis database client
|
|
RedisX v1.0
A simple, light-weight Redis database client
|
A free, simple, and light-weight C/C++ Redis / Valkey client library.
Author: Attila Kovacs
Updated for 1.0 and later releases.
RedisX is a free, light-weight Redis client library for C/C++. It works with Redis forks / clones like Dragonfly or Valkey also. It supports both interactive and batch queries, managing and processing subscriptions, atomic execution blocks, and LUA script loading. It can be used with multiple Redis servers simultaneously also. RedisX is free to use, in any way you like, without licensing restrictions.
While there are other C/C++ Redis clients available, this one is C99 compatible, and hence can be used on older platforms also. It is also small and fast, but still capable and versatile.
Rather than providing high-level support for every possible Redis command (which would be challenging given the pace new commands are being introduced all the time), it provides a basic framework for synchronous and asynchronous queries, with some higher-level functions, such as for managing key/value storage types (including hash tables) and PUB/SUB. Future releases may add further higher-level functionality based on demand for such features.
The RedisX library was created, and is maintained, by Attila Kovács at the Center for Astrophysics | Harvard & Smithsonian, and it is available through the Smithsonian/redisx repository on GitHub.
Below is a minimal example of a program snippet, which connects to a Redis server (without authentication) and runs a simple PING comand with a message, printing out the result on the console, while also following best practices of checking for errors, and handling them – in this case by printing informative error messages:
You may find more details about each step further below. And, of course there are a lot more options and features, that allow you to customize your interactions with a Redis / Valkey server. But the basic principle, and the steps, follow the same pattern in general:
And at every step, you should check for and handle errors as appropriate.
| Feature | supported | comments |
|---|---|---|
| concurrent Redis instances | yes | You can manage and use multiple Redis servers simultaneously |
| thread safe (MT-safe) | yes | synchronized + async calls with locking |
| connect / disconnect hooks | yes | user-defined callbacks |
| socket customization | yes | (optional) user-defined timeout, buffer size and/or callback |
| custom socket error handling | yes | (optional) user-defined callback |
| RESP to JSON | yes | via xchange library |
| RESP to structured data | yes | via xchange library |
| debug error tracing | yes | via xSetDebug() |
| command-line client | yes | redisx-cli |
| Redis / Valkey Feature | supported | comments |
|---|---|---|
| user authentication | yes | via HELLO if protocol is set, otherwise via AUTH |
RESP3 / HELLO support | yes | (optional) if specific protocol is set |
| interactive queries | yes | dedicated (low-latency) client |
| pipelined (batch) processing | yes | dedicated (high-bandwidth) client / user-defined callback |
| PUB/SUB support | yes | dedicated client / user callbacks / subscription management |
| push messages | yes | (optional) user-defined callback |
| attributes | yes | (optional) on demand |
| Sentinel support | yes | help me test it |
| cluster support | yes | help me test it |
| ... MOVED redirection | yes | automatic for interactive transactions |
| ... ASK redirection | yes | automatic for interactive transactions |
| TLS support | yes | help me test it |
The Smithsonian/xchange library is both a build and a runtime dependency of RedisX. There are some optional dependencies, depending on the build configuration:
openssl-devel on RPM-based, or libssl-dev on Debian-based Linux) to build with TLS support.libomp-devel on RPM-based, or libomp-dev on Debian-based Linux) is needed to build with parallelization support (for parallelized cluster connection / disconnection).Additionally redisx-cli has the following dependencies on standard GNU/POSIX libraries:
popt-devel on RPM-based, or libpopt-dev on Debian based Linux).libbsd-devel on RPM-based, or libbsd-dev on Debian based Linux).readline-devel on RPM based, or libreadline-dev on Debian based Linux).The RedisX library can be built either as a shared (libredisx.so[.1]) or as a static (libredisx.a) library, depending on what suits your needs best.
You can configure the build, either by editing config.mk or else by defining the relevant environment variables prior to invoking make. The following build variables can be configured:
CC: The C compiler to use (default: gcc).CPPFLAGS: C preprocessor flags, such as externally defined compiler constants.CFLAGS: Flags to pass onto the C compiler (default: -g -Os -Wall). Note, -Iinclude will be added automatically.CSTANDARD: Optionally, specify the C standard to compile for, e.g. c11 to compile for the C11 standard. If defined then -std=$(CSTANDARD) is added to CFLAGS automatically. Note, that some pattern matching functions, which use fnmatch() may not be available in the C99 standard, but can still be enabled if you add -D_POSIX_C_SOURCE=200112L to CPPFLAGS also, and link with a LIBC version that supports the POSIX.1-2001 standard.WEXTRA: If set to 1, -Wextra is added to CFLAGS automatically.FORTIFY: If set it will set the _FORTIFY_SOURCE macro to the specified value (gcc supports values 1 through 3). It affords varying levels of extra compile time / runtime checks.LDFLAGS: Extra linker flags (default is not set). Note, -lm -lpthread -lxchange will be added automatically.WITH_OPENMP: If set to 1, we will compile and link with OpenMP (i.e., -fopenmp is added to both CFLAGS and LDFLAGS automatically). If not explicitly defined, it will be set automatically if libomp is available.WITH_TLS: If set to 1, we will build with TLS support via OpenSSL (And -lssl is added to LDFLAGS automatically). If not explicitly defined, it will be set automatically if libssl is available.FNMATCH_C: (for older platforms) Optional full path to fnmatch.c to use. Normally fnmatch() (POSIX-1.2001) is part of LIBC, but for older systems, or if CSTANDARD is set to c99, is will be assumed as not present, and hence the redisxDeleteEntries() function will not be available, unless you set FNMATCH_C to point to a suitable fnmatch.c implementation to link against.CHECKEXTRA: Extra options to pass to cppcheck for the make check targetDOXYGEN: Specify the doxygen executable to use for generating documentation. If not set (default), make will use doxygen in your PATH (if any). You can also set it to none to disable document generation and the checking for a usable doxygen version entirely.XCHANGE: If the Smithsonian/xchange library is not installed on your system (e.g. under /usr) set XCHANGE to where the distribution can be found. The build will expect to find xchange.h under $(XCHANGE)/include and libxchange.so / libxchange.a under $(XCHANGE)/lib or else in the default LD_LIBRARY_PATH.STATICLINK: Set to 1 to prefer linking tools statically against libredisx.a. (It may still link dynamically if libredisx.so is also available.)After configuring, you can simply run make, which will build the shared (lib/libredisx.so[.1]) and static (lib/libredisx.a) libraries, local HTML documentation (provided doxygen is available), and performs static analysis via the check target. Or, you may build just the components you are interested in, by specifying the desired make target(s). (You can use make help to get a summary of the available make targets).
After building the library you can install the above components to the desired locations on your system. For a system-wide install you may simply run:
Or, to install in some other locations, you may set a prefix and/or DESTDIR. For example, to install under /opt instead, you can:
Or, to stage the installation (to /usr) under a 'build root':
The RedisX library provides its own command-line tool, called redisx-cli. It works very similar to redis-cli, except that our client has somewhat fewer bells and whistles.
will print:
provided it successfully connected to the Redis / Valkey server on localhost. (Otherwise it will print an error and a trace). It can also be used in interactive mode if no Redis command arguments are supplied. And, you can run redisx-cli --help to see what options are available, and you can also consult the redis-cli documentation for the same general description and usage (so far as our implementation supports it).
Provided you have installed the shared (libredisx.so and libxchange.so) or static (libredisx.a and libxchange.a) libraries in a location that is in your LD_LIBRARY_PATH (e.g. in /usr/lib or /usr/local/lib) you can simply link your program using the -lredisx -lxchange flags. Your Makefile may look like:
(Or, you might simply add -lredisx -lxchange to LDFLAGS and use a more standard recipe.) And, in if you installed the RedisX and/or xchange libraries elsewhere, you can simply add their location(s) to LD_LIBRARY_PATH prior to linking.
The library maintains up to three separate connections (channels) for each separate Redis server instance used: (1) an interactive client for sequential round-trip transactions, (2) a pipeline client for bulk queries and asynchronous background processing, and (3) a subscription client for PUB/SUB requests and notifications. The interactive client is always connected, the pipeline client is connected only if explicitly requested at the time of establishing the server connection, while the subscription client is connected only as needed.
The first step is to create a Redis object, with the server name or IP address.
Alternatively, instead of redisxInit() above you may initialize the client for a high-availability configuration with a set of Redis Sentinel servers, using redisxInitSentinel(), e.g.:
After successful initialization, you may proceed with the configuration the same way as for the regular standalone server connection.
One thing to keep in mind about Sentinel is that once the connection to the master is established, it works just like a regular server connection, including the possibility of that connection being broken. It is up to the application to initiate reconnection and recovery as appropriate in case of errors. (See more in on Reconnecting further below).
The Sentinel support is still experimental and requires testing. You can help by submitting bug reports in the GitHub repository.
Before connecting to the Redis server, you may configure the database authentication (if any):
You can also set the RESP protocol to use (provided your server is compatible with Redis 6 or later):
The above call will use the HELLO command (since Redis 6) upon connecting. If you do not set the protocol, HELLO will not be used, and RESP2 will be assumed – which is best for older servers (Redis <6). (Note, that you can always check the actual protocol used after connecting, using redisxGetProtocol()). Note, that after connecting, you may retrieve the set of server properties sent in response to HELLO using redisxGetHelloData().
You can also set a timeout for the interative transactions, such as:
You may also select the database index to use now (or later, after connecting), if not the default (index 0):
Note, that you can switch the database index any time, with the caveat that it's not possible to change it for the subscription client when there are active subscriptions.
We provide (experimental) support for TLS (see the Redis docs on TLS support). Simply configure the necessary certificates, keys, and cypher parameters as needed, e.g.:
The TLS support is still experimental and requires testing. You can help by submitting bug reports in the GitHub repository.
You might also tweak the socket options used for clients, if you find the socket defaults sub-optimal for your application:
If you want, you can perform further customization of the client sockets via a user-defined callback function, e.g.:
which you can then apply to your Redis instance as:
The user of the RedisX library might want to know when connections to the server are established, or when clients get disconnected (either as intended or as a result of errors), and may want to perform some configuration or clean-up accordingly. For this reason, the library provides support for connection 'hooks' – that is custom functions that are called in the even of connecting to or disconnecting from a Redis server.
Here is an example of a connection hook, which simply prints a message about the connection to the console.
And, it can be added to a Redis instance, between the redisxInit() and the redisxConnect() calls.
You may add multiple callbacks. All of them will be called (in the same order as added) when connection is established. You may also remove specific connection callbacks via redisxRemoveConnectHook() if you now longer want a particular function to be called any more in the even.
The same goes for disconnect hooks, using redisxAddDisconnectHook() / redisxRemoveDisconnectHook() instead.
Once configured, you can connect to the server as:
The above will establish both an interactive connection and a pipelined connection client, for processing both synchronous and asynchronous requests (and responses). For Sentinel configurations, it will return with X_SUCCESS only after having located and connected to the master server, and confirmed that it is indeed the master.
When you are done with a specific Redis server, you should disconnect from it:
And then to free up all resources used by the Redis instance, you might also call
Reconnections to the Redis servers are never automatic, and there is no automatic failover for RedisX clients (there are good reasons for that). It is up to you to decide when to reconnect and how exactly. For example, the application may reconnect to the same or different server (including Sentinel), and perform a set of necessary recovery steps, to continue where things were left off on the previous connection, such as:
Redis queries are sent as strings, according the the specification of the Redis protocol. All responses sent back by the server using the RESP protocol. Specifically, Redis uses version 2 of the RESP protocol (a.k.a. RESP2) by default, with optional support for the newer RESP3 introduced in Redis version 6.0. The RedisX library provides support for both RESP2 and RESP3.
The simplest way for running a few Redis queries is to do it in interactive mode:
The redisxRequest() sends a command with up to three arguments. If the command takes fewer than 3 parameters, then the remaining ones must be set to NULL. This function thus offers a simple interface for running most basic sequential queries. In cases where 3 parameters are not sufficient, you may use redisxArrayRequest() instead, e.g.:
The 3rd argument in the list is an optional int[] array defining the individual string lengths of the parameters (if need be, or else readily available). Here, we used NULL instead, which will use strlen() on each supplied string-terminated parameter to determine its length automatically. Specifying the length may be necessary if the individual parameters are not 0-terminated strings.
In interactive mode, each request is sent to the Redis server, and the response is collected before the call returns with that response (or NULL if there was an error).
Redis 6 introduced the possibility of sending optional attributes along with responses, using the RESP3 protocol. These attributes are not returned to users by default, in line with the RESP3 protocol specification. Rather, they are available on demand, after the response to a request is received. You may retrieve the attributes to interactive requests after the redisxRequest() or redisxArrayRequest() queries, using redisxGetAttributes(), e.g.:
Redis 6 introduced out-of-band push notifications along with RESP3. It allows the server to send messages to any connected client that are not in response to a query. For example, Redis 6 allows CLIENT TRACKING to use such push notifications (e.g. INVALIDATE foo), to notify connected clients when a watched variable has been updated from somewhere else.
RedisX allows you to specify a custom callback RedisPushProcessor function to handle such push notifications, e.g.:
Then you can activate the processing of push notifications with redisxSetPushProcessor(). You can specify the optional additional data that you want to pass along to the push processor function – just make sure that the data has a sufficient scope / lifetime such that it is valid at all times while push messages are being processed. E.g.
There are some things to look out for in your RedisPushProcessor implementation:
All responses coming from the Redis server are represented by a dynamically allocated RESP type (defined in redisx.h) structure.
whose contents are:
RESP type | Redis ID | n | value cast in C |
|---|---|---|---|
RESP_ARRAY | * | number of RESP * pointers | (RESP **) |
RESP_INT | : | integer return value | (void) |
RESP_SIMPLE_STRING | + | string length | (char *) |
RESP_ERROR | - | total string length | (char *) |
RESP_BULK_STRING | $ | string length or -1 if NULL | (char *) |
RESP3_NULL | _ | 0 | (void) |
RESP3_BOOLEAN | # | 1 if true, 0 if false | (void) |
RESP3_DOUBLE | , | unused | (double *) |
RESP3_BIG_NUMBER | ( | string representation length | (char *) |
RESP3_BLOB_ERROR | ! | total string length | (char *) |
RESP3_VERBATIM_TEXT | = | text length (incl. type) | (char *) |
RESP3_SET | ~ | number of RESP * pointers | (RESP *) |
RESP3_MAP | % | number of key / value pairs | (RedisMap *) |
RESP3_ATTRIBUTE | \| | number of key / value pairs | (RedisMap *) |
RESP3_PUSH | > | number of RESP * pointers | (RESP **) |
Each RESP has a type (e.g. RESP_SIMPLE_STRING), an integer value n, and a value pointer to further data. If the type is RESP_INT, then n represents the actual return value (and the value pointer is not used). For string type values n is the number of characters in the string value (not including termination), while for RESP_ARRAY types the value is a pointer to an embedded RESP array and n is the number of elements in that.
To help with deciding what cast to use for a given value field of the RESP data structure, we provide the convenience methods redisxIsScalarType(), redisxIsStringType(), redisxIsArrayType(), and redisxIsMapType() functions.
You can check that two RESP data structures are equivalent with redisxIsEqualRESP(RESP *a, RESP *b).
You may also check the integrity of a RESP using redisxCheckRESP(). Since RESP data is dynamically allocated, the user is responsible for discarding them once they are no longer needed, e.g. by calling redisxDestroyRESP(). The two steps may be combined to automatically discard invalid or unexpected RESP data in a single step by calling redisxCheckDestroyRESP().
Before destroying a RESP structure, the caller may want to dereference values within it if they are to be used as is (without making copies), e.g.:
Note, that you can convert a RESP to an XField, and/or to JSON representation using the redisxRESP2XField() and redisxRESP2JSON() functions, e.g.:
All RESP can be represented in JSON format. This is trivial for map entries, which have strings as their keywords – which is the case for all RESP sent by Redis. And, it is also possible for map entries with non-string keys, albeit via a more tedious (and less standard) JSON representation, stored under the .non-string-keys keyword.
Key/value pairs are the bread and butter of Redis. They come in two varieties: (1) there are top-level key-value pairs, and (2) there are key-value pairs organized into hash tables, where the table name is a top-level key, but the fields in the table are not. The RedisX library offers a unified approach for dealing with key/value pairs, whether they are top level or hash-tables. Simply, a table name NULL is used to refer to top-level keys.
Retrieving individual keyed values is simple:
The same goes for top-level keyed values, using NULL for the hash table name:
Alternatively, you can get the value as an undigested RESP, using redisxGetValue() instead, which allows you to check and inspect the response in more detail (e.g. to check for potential errors, or unexpected response types).
Setting values is straightforward also:
It's worth noting here, that values in Redis are always represented as strings, hence non-string data, such as floating-point values, must be converted to strings first. Additionally, the redisxSetValue() function works with 0-terminated string values only, but Redis allows storing unterminated byte sequences of known length also. If you find that you need to store an unterminated string (such as a binary sequence) as a value, you may just use the lower-level redisxArrayRequest() instead to process a Redis SET or HSET command with explicit byte-length specifications.
In the above example we have set the value using the interactive client to Redis, which means that the call will return only after confirmation or result is received back from the server. As such, a subsequent redisxGetValue() of the same table/key will be guaranteed to return the updated value always.
However, we could have set the new value asynchronously over the pipeline connection (by using TRUE as the last argument). In that case, the call will return as soon as the request was sent to Redis (but not confirmed, nor possibly transmitted yet!). As such, a subsequent redisxGetValue() on the same key/value field may race the request in transit, and may return the previous value on occasion. So, it's important to remember that while pipelining can make setting multiple Redis fields very efficient, we have to be careful about retrieving the same values afterwards from the same program thread. (Arguably, however, there should never be a need to query values we set ourselves, since we readily know what they are.)
Finally, if you want to set values for multiple fields in a Redis hash table atomically, you may use redisxMultiSet(), which provides a high-level interface to the Redis HMSET command.
The functions redisxGetKeys() and redisxGetTable() allow to return the set of Redis keywords or all key/value pairs in a table atomically. However, these commands can be computationally expensive for large tables and/or many top-level keywords, which means that the Redis server may block for undesirably long times while the result is computed.
This is where scanning offers a less selfish (hence much preferred) alternative. Rather than returning all the keys or key/value pairs contained in a table atomically at once, it allows to do it bit by bit with byte-sized individual transactions that are guaranteed to not block the Redis server long, so it may remain responsive to other queries also. For the caller the result is the same (notwithstanding the atomicity), except that the result is computed via a series of quick Redis queries rather than with one potentially very expensive query.
For example, to retrieve all top-level Redis keys, sorted alphabetically, using the scanning approach, you may write something like:
Or, to retrieve the values from a hash table for a set of keywords that match a glob pattern:
Finally, you may use redisxSetScanCount() to tune just how many results should individual scan queries should return (but only if you are really itching to tweak it). Please refer to the Redis documentation on the behavior of the SCAN and HSCAN commands to learn more.
It is simple to send messages to subscribers of a given channel:
The last argument is an optional string length, if readily available, or if sending a byte sequence that is not null-terminated. If zero is used for the length, as in the example above, it will automatically determine the length of the 0-terminated string message using strlen().
Alternatively, you may use the redisxPublishAsync() instead if you want to publish on a subscription client to which you have already have exclusive access (e.g. after an appropriate redisxLockConnected() call).
Subscriptions work conceptually similarly to pipelined requests. To process incoming messages you need to first specify one or more RedisSubscriberCall functions, which will process PUB/SUB notifications automatically, in the background, as soon as they are received. Each RedisSubscriberCall can pre-filter the channels for which it receives notifications, by defining a channel stem. This way, the given processor function won't even be invoked if a notification on a completely different channel arrives. Still, each RedisSubscriberCall implementation should further check the notifying channel name as appropriate to ensure that it is in fact qualified to deal with a given message.
Here is an example RedisSubscriberCall implementation to process messages:
There are some basic rules (best practices) for message processing. They should be fast, and never block for extended periods. If extensive processing is required, or may need to wait extensively for some resource or mutex locking, then its best that the processing function simply places the incoming message onto a queue, and let a separate background thread do the heavy lifting without holding up the subscription processing of other callback routines, or without losing responsiveness to other incoming messages.
Also, it is important that the call should never attempt to modify or call free() on the supplied string arguments, since that would interfere with other subscriber calls.
Once the function is defined, you can activate it via:
We should also start subscribing to specific channels and/or channel patterns.
The redisxSubscribe() function will translate to either a Redis PSUBSCRIBE or SUBSCRIBE command, depending on whether the pattern argument contains globbing patterns or not (respectively).
Now, we are capturing and processing all messages published to channels whose name begins with "event:", using our custom my_event_processor function.
To end the subscription, we trace back the same steps by calling redisxUnsubscribe() to stop receiving further messages to the subscription channel or pattern, and by removing the my_event_procesor subscriber function as appropriate (provided no other subscription needs it) via redisxRemoveSubscriber().
Sometimes you may want to execute a series of Redis command atomically, such that nothing else may alter the database while the set of commands execute, so that related values are always in a coherent state. For example, you want to set or query a collection of related variables so they change together and are reported together. You have two choices. (1) you can execute the Redis commands in an execution block, or else (2) load a LUA script onto the Redis server and call it with some parameters (possibly many times over).
Execution blocks offer a fairly simple way of bunching together a set of Redis commands that need to be executed atomically. Such an execution block in RedisX may look something like:
If at any point things don't go according to plan in the middle of the block, you can call redisAbortBlockAsync() to abort and discard all prior commands submitted in the execution block already. It is important to remember that every time you call redisxStartBlockAsync(), you must call either redisxExecBlockAsync() to execute it or else redisxAbortBlockAsync() to discard it. Failure to do so, will effectively end you up with a hung Redis client.
LUA scripting offers a more capable version of executing complex routines on the Redis server. LUA is a scripting language akin to python, and allows you to add extra logic, string manipulation etc. to your Redis queries. Best of all, once you upload the script to the server, it can reduce network traffic significantly by not having to repeatedly submit the same set of Redis commands every single time. LUA scripts also get executed very efficiently on the server, and produce only the result you want/need without returning unnecessary intermediates.
Assuming you have prepared your LUA script appropriately, you can upload it to the Redis server as:
Redis will refer to the script by its SHA1 sum, so it's important keep a record of it. You'll call the script with its SHA1 sum, a set of Redis keys the script may use, and a set of other parameters it might need.
Or, you can use redisxRunScriptAsync() instead to send the request to run the script, and then collect the response later, as appropriate.
One thing to keep in mind about LUA scripts is that they are not fully persistent. They will be lost each time the Redis server is restarted.
Functions, introduced in Redis 7, offer another evolutionary step over the LUA scripting described above. Unlike scripts, functions are persistent and they can be called by name rather than a cryptic SHA1 sum. Otherwise, they offer more or less the same functionality as scripts. RedisX does not currently have a built-in high-level support for managing and calling user-defined functions, but it is a feature that may be added in the not-too-distant future. Stay tuned.
Sometimes you might want to micro manage how requests are sent and responses to them are received. RedisX provides a set of asynchronous client functions that do that. (You've seen these already further above in the Pipelined transaction section.) These functions should be called with the specific client's mutex locked, to ensure that other threads do not interfere with your sequence of requests and responses. E.g.:
While you have the exclusive lock you may send any number of requests, e.g. via redisxSendRequestAsync() and/or redixSendArrayRequestAsync(). Then collect replies either with redisxReadReplyAsync() or else redisxIgnoreReplyAsync(). For example, the basic anatomy of sending a single request and then receiving a response, while we have exclusive access to the client, might look something like this:
For the best performance, you may want to leave the processing of the replies until after you unlock the client. I.e., you only block other threads from accessing the client while you send off the requests and collect the corresponding responses. You can then analyze the responses at your leisure outside of the mutexed section.
In some cases you may be OK with just firing off some Redis commands, without necessarily caring about responses. Rather than ignoring the replies with redisxIgnoreReplyAsync() you might call redisxSkipReplyAsync() instead before redisxSendRequestAsync() to instruct Redis to not even bother about sending a response to your request (it saves time and network bandwidth!):
Of course you can build up arbitrarily complex set of queries and deal with a set of responses in different ways. Do what works best for your application.
As of Redis 6, the server might send ancillary data along with replies, if the RESP3 protocol is used. These are collected together with the expected responses. However, these optional attributes are not returned to the user automatically. Instead, the user may retrieve attributes directly after getting a response from redisxReadReplyAsync() using redisxGetAttributesAsync(). And, attributes that were received previously can be discarded with redisxClearAttributesAsync(). For example,
Depending on round-trip times over the network, interactive queries may be suitable for running up to a few thousand queries per second. For higher throughput (up to ~1 million Redis transactions per second) you may need to access the Redis database in pipelined mode. RedisX provides a dedicated pipeline client/channel to the Redis server (provided the option to enable it has been used when redixConnect() was called).
In pipeline mode, requests are sent to the Redis server over the pipeline client in quick succession, without waiting for responses to return for each one individually. Responses are processed in the background by a designated callback function (or else discarded if no callback function has been set). This is what a callback function looks like:
It is important to note that the processing function should not call free on the RESP pointer argument, but it may dereference and use parts of it as appropriate (just remember to set the bits referenced elsewhere to NULL so they do not get destroyed when the pipeline listener destroys the RESP after your function is done processing it). Before sending the pipelined requests, the user first needs to specify the function to process the responses, e.g.:
Request are sent via the redisxSendRequestAsync() and redisxSendArrayRequestAsync() functions. Note again, the Async naming, which indicates the asynchronous nature of this calls – and which indicates that these functions should be called with the appropriate mutex locked to prevent concurrency issues, and to maintain a predictable order (very important!) for processing the responses.
It is important to remember that on the pipeline client you should never try to process responses directly from the same function from which commands are being sent. That's what the interactive connection is for. Pipeline responses are always processed by a background thread (or, if you don't specify your callback function they will be discarded). The only thing your callback function can count on is that the same number of responses will be received as the number of asynchronous requests that were sent out, and that the responses arrive in the same order as the order in which the requests were sent.
It is up to you and your callback function to keep track of what responses are expected and in what order. Some best practices to help deal with pipeline responses are summarized here:
redisxSkipReplyAsync() prior to sending pipeline requests for which you do not need a response. (This way your callback does not have to deal with unnecessary responses at all.HGET commands, your FIFO should have a record that specifies that a bulk string response is expected, and a pointer to data which is used to store the returned value – so that you pipeline response processing callback function can check that the response is the expected type (and size) and knows to assign/process the response appropriately to your application data.PING/ECHO commands to section your responses, or to provide directives to your pipeline response processor function. You can tag them uniquely so that the echoed responses can be parsed and interpreted by your callback function. For example, you may send a PING/ECHO commands to Redis with the tag "@my_resp_processor: START sequence A", or something else meaningful that you can uniquely distinguish from all other responses that you might receive.RedisX optimizes the pipeline client for high throughput (bandwidth), whereas the interactive and subscription clients are optimized for low-latency, at the socket level.
RedisX provides support for Redis clusters also. In cluster configuration the database is distributed over a collection of servers, each node of which serves only a subset of the Redis keys.
The support for Redis clusters is still experimental and requires testing. You can help by submitting bug reports in the GitHub repository.
Specifically, start by configuring a known node of the cluster as usual for a single Redis server, setting authentication, socket configuration, callbacks etc.:
Next, you can use the known node to obtain the cluster configuration:
The above will query the cluster configuration from the node (the node need not be explicitly connected prior to the initialization, and will be returned in the same connection state as before). The cluster will inherit the configuration of the node, such as pipelining, socket configuration authentication, protocol, and callbacks, from the configuring node – all but the database index, which is always 0 for clusters.
If the initialization fails on a particular node, you might try other known nodes until one of then succeeds. (You might use redisxSetHostName() and redisxSetPort() on the configured Redis instance to update the address of the configuring node, while leaving other configuration settings intact.)
Once the cluster is configured, you may discard the configuring node instance, unless you need it specifically for other reasons.
You can start using the cluster right away. You can obtain a connected Redis instance for a given key using redisxClusterGetShard(), e.g.:
The interactive queries handle both MOVED and ASK redirections automatically. However, asynchronous queries do not since they return before receiving a response. Thus, when using redisxReadReplyAsync() later to process replies, you should check for redirections:
As a matter a best practice you should never assume that a given keyword is persistently served by the same shard. Rather, you should obtain the current shard for the key each time you want to use it with the cluster, and always check for errors on shard requests, and repeat failed requests on a newly obtained shard if necessary.
Finally, when you are done using the cluster, simply discard it:
In the above example we have shown one way you might check for errors that result from cluster being reconfigured on-the-fly, using redisxClusterMoved() and/or redisxClusterIsMigrating() on the RESP reply obtained from the shard.
Equivalently, you might use redisxCheckRESP() or redisxCheckDestroyRESP() also for detecting a cluster reconfiguration. Both of these will return a designated REDIS_MOVED or REDIS_MIGRATING error code if the keyword has moved or is migrating, respectively, to another node, e.g.:
To help manage redirection responses for asynchronous requests, we provide redisxClusterGetRedirection() to obtain the redirected Redis instance based on the redirection RESP. Once the redirected cluster shard is identified you may either resubmit the same query as before (e.h. with redisxSendArrayRequestAsync()) if MOVED, or else repeat the query via an interactive ASKING directive using redisxClusterAskMigrating().
The redisxClusterGetShard() will automatically connect the associated shard, if not already connected. Thus, you do not need to explicitly connect to the cluster or its shards. However, in some cases you might want to connect all shards before running queries to eliminate the connection overhead during use. If so, you can call redisxClusterConnect() to explicitly connect to all shards before using the cluster. Similarly, you can also explicitly disconnect from all connected shards using redisxClusterDisconnect(), e.g. to close unnecessary sockets. You may continue to use the cluster after calling redisxClusterDisconnect(), as successive calls to redisxClusterGetShard() will continue to reconnect the shards as needed automatically.
The principal error handling of RedisX is an extension of that of xchange, with further error codes defined in redisx.h. The RedisX functions that return an error status (either directly, or into the integer designated by a pointer argument), can be inspected by redisxErrorDescription(), e.g.:
In addition you can define your own handler function to deal with transmission (send/receive) errors, by defining your own RedisErrorHandler function, such as:
Then activate it as:
After that, every time there is an error with sending or receiving packets over the network to any of the Redis clients used, your handler will report it the way you want it.
There are multiple ways the user may get informed when errors happen at the socket read / write level:
X_NO_SERVICE error code or NULL returns, plus errno), andIt is up to you to decide which of the following method(s) you wish to rely on to detect broken connections and act as appropriate. To help with your decisions, below is a step-by-step outline of how RedisX handles errors originating from the socket level of clients, indicating also the points at which users are notified by each of the above mentioned methods.
Async call on the affected client as appropriate, but the error handler should not attempt to release the exclusive lock on the client or call synchronized functions on the Redis instance or the affected client. The background processing of replies (on the pipeline and/or subscription clients) is still active at this stage.errno being EAGAIN or EWOULDBLOCK), nothing changes at this stage. However, if the error is persistent, the client will be disabled and reset, and subsequent read or write calls will fail immediately. Any disconnection hooks will be called also as the client is disconnected. The background processing of server replies (on the pipeline and subscriptions channels) will stop soon after the disconnection is initiated.X_NO_SERVICE, or X_TIMEDOUT, or else NULL. The application should check return values (and/or errno) as appropriate.You can enable verbose output of the RedisX library with xSetVerbose(boolean). When enabled, it will produce status messages to stderrso you can follow what's going on. In addition (or alternatively), you can enable debug messages with xSetDebug(boolean). When enabled, all errors encountered by the library (such as invalid arguments passed) will be printed to stderr, including call traces, so you can walk back to see where the error may have originated from. (You can also enable debug messages by default by defining the DEBUG constant for the compiler, e.g. by adding -DDEBUG to CFLAGS prior to calling make).
In addition, you can use redisxDebugTraffic(boolean) to debug low-level traffic to/from the Redis server, which prints excerpts of all incoming and outgoing messages from/to Redis to stderr.
For helping to debug your application, the xchange library provides two macros: xvprintf() and xdprintf(), for printing verbose and debug messages to stderr. Both work just like printf(), but they are conditional on verbosity being enabled via xSetVerbose(boolean) and xSetDebug(boolean), respectively. Applications using RedisX may use these macros to produce their own verbose and/or debugging outputs conditional on the same global settings.
Some obvious ways the library could evolve and grow in the not too distant future:
If you have an idea for a must have feature, please let me (Attila) know. Pull requests, for new features or fixes to existing ones, are especially welcome!
Copyright (C) 2025 Attila Kovács
1.9.8