spark publish

The publish command is used to publish data to a topic on an AMPS server.

Common Options - spark publish

Option	Definition
server (required)	AMPS server to connect to.
topic (required)	Topic to publish to.
delimiter	Decimal value of message separator character (default 10).
delta	Use delta publish (sends a delta_publish command to AMPS).
file	File to publish messages from, stdin when omitted. AMPS spark CLI interprets each line in the input as a message. The file provided to this argument can be either uncompressed or compressed in ZIP format. If a ZIP file is provided, it must contain one message per file in the same structure produced by amps_sow_dump -z. The ZIP format requires an archive where the first message payload is stored in a file named 1, the second in a file named 2, and so on, with all message files located in the ZIP's top-level directory.
proto	Protocol to use. In this release, AMPS spark CLI supports amps, fix, nvfix and xml. Defaults to amps. AMPS spark CLI also supports json as a synonym for amps in this release.
rate	Messages to publish per second. This is a decimal value, so values less than 1 can be provided to create a delay of more than a second between messages. '.25' implies 1 message every 4 seconds. '1000' implies 1000 messages per second.
secure (applies to SSL connections)	Specifies whether to use an SSL-secured connection (tcps URI scheme). Values: false/0/no or true/1/yes Default: false
type	For protocols and transports that accept multiple message types on a given transport, specifies the message type to use.
uriopts	Custom connection URI parameters to be passed in the URI query string. For example: tcp_nodelay=true&tcp_sndbuf=8192
urischeme (applies to SSL connections)	Allows a custom URI scheme to be specified. When both secure and urischeme are specified, urischeme takes precedence.

Examples

The examples shown below will demonstrate how to publish records to AMPS using the AMPS spark CLI in one of the three following ways: a single record, a python script or by file.

Publish a Single Message

%> echo '{ "id" : 1, "data": "hello, world!" }' |  \
   ./spark publish -server localhost:9007 -type json -topic order

   total messages published: 1 (50.00/s)

In the example above, a single record is published to AMPS using the echo command. If you are comfortable with creating records by hand this is a simple and effective way to test publishing in AMPS.

The JSON message is published to the topic order on the AMPS instance. Assuming the order topic is configured as a SOW topic, this publish can be followed with a sow command in AMPS spark CLI to test if the record was indeed published to the order topic.

Publish using Python

%> python -c "for n in range(100): print('{\"id\":%d}' % n)" | \
      ./spark publish -topic disorder -type json -rate 50 \
      -server localhost:9007

   total messages published: 100 (50.00/s)

In the example above, the -c flag is used to pass in a simple loop and print command to the python interpreter and have it print the results to stdout.

The python script generates 100 JSON messages of the form {"id":0}, {"id":1} ... {"id":99}. The output of this command is then piped to AMPS spark CLI using the | character, which will publish the messages to the disorder topic inside the AMPS instance.

Publish from a File

%> ./spark publish -server localhost:9007 -type json -topic chaos \
   -file data.json

   total messages published: 50 (12000.00/s)

Generating a file of test data is a common way to test AMPS functionality. The example above demonstrates how to publish a file of data to the topic chaos in an AMPS server. AMPS spark CLI interprets each line of a text file as a distinct message.

For the full range of options, run spark help publish. See AMPS spark CLI Authentication and AMPS spark CLI Support for TCPS for shared connection settings.