How to Create Effective Blockchain Queries using Hyperledger Composer
I have always had a particular interest in the IBM Hyperledger and IBM Blockchain platforms for creating various business models.
If you go through my blog carefully, then you will see that I have already written and posted a couple of previous articles surrounding IBM Hyperledger and Blockchain services.
- How to deploy a Business Network Archive (.bna) file to your IBM Blockchain Platform
- How to build a small blockchain application using the IBM BLUEMIX
- How to setup an MSP channel using the IBM Hyperledger fabric
So just to add into this growing list of IBM posts, in this article I will give you a step by step comprehensive run through on how to create effective Blockchain queries using IBM Hyperledger for your business.
Importance of Blockchain queries
Creating effective and working blockchain queries has wide importance when it comes to application developers. Here querying resources, such as transaction data and assets in blocks on a blockchain, can be a potentially time consuming and a very circuitous task.
So when it comes to creating business blockchain queries, you may encounter the following questions or rather doubts in your mind.
- How should I compose an effective business query that can be understood smoothly by the blockchain?
- How do I get the results?
- How do I consume the results?
Many people consider the sets of diverse and disparate nature of data in varying formats, transforming and then extracting results and showcase them in a presentable way as a burden.
This is where IMB Hyperledger comes into play in making your task of submitting and then handling query results simple and clean.
IBM Hyperledger Composition
Hyperledger Composer is based on Hyperledger Fabric v1.0, but prior to Hyperledger Fabric v1.0, queries required IDs generated by Hyperledger Fabric.
Now with Hyperledger Composer's rich query capability and Hyperledger Fabric v1.0, you can query resources using any attribute or variable parameters. By using queries with parameters, you don't have to construct complex ad hoc queries, and you can construct template queries, setting the parameters for each use.
Common Blockchain queries
Consider these logical queries, for example:
- "Show me the list of commodities created by Trader A."
- "Show me the list of commodities of type EURONEXT created by Trader B in the last week."
- "Show me the history of changes to the ledger for a particular Commodity instance, by a particular Trader, and for a specified time window."
Translation of these queries into "Blockchain Speak"
Using Hyperledger Composer's rich Query feature allows you to compose powerful queries like the one’s mentioned above. This unique Query feature bridges the gap, Hence you will no longer have to build the syntax, parse, convert, and validate the query. You won’t even have to build up its types, and the results you get back from a Hyperledger Fabric blockchain. Even also would not have to worry about ensuring a consistent format for your Blockchain.
Another big plus is that, Hyperledger Composer's Query language eliminates these hassles, thus making it further easy to define queries, in simple notation. This efficiently enables your application to take actions based on business-defined criteria. Once this is done, you can then easily consume the queries directly in your application. This can also be done via the generated query REST APIs.
Recommendations
I would strongly recommend you to grab the monthly roundup that will allow you to use the best free tools, training, and community resources to help you put blockchain to work and achieve the best results.
Tutorial
The Hyperledger Composer Query language is powerful enough to return efficient results using a blockchain resource's attributes. To explain how it is basically done, we will take an existing trade-network sample deployed using Hyperledger Composer as an example and show how to use rich queries with a REST server. The trade-network sample simply shows the transfer of ownership of commodities between two different traders.
Through this same procedure, we will also show you how to perform operations on result sets, such as updating or removing assets using a transaction processor.
I would strongly recommend you to work through this tutorial as per the steps and software listed below. Please do not skip any step in between. You will also require a working knowledge of JSON and basic query definitions.
Once you have gone through the tutorial, you can easily use Hyperledger Composer to build your network and try out your own queries.
Prerequisites and Software requirements
- Hyperledger Composer development tools. We will show you how to install them in the next section.
- Hyperledger Composer command line (composer-cli)
- Hyperledger Composer REST server (composer-rest-server)
- A running Hyperledger Fabric runtime (v1 GA edition). We will give you the guidelines for downloading and starting the Hyperledger Fabric environment in the next section.
CLINK ON THIS LINK for installing the development tools.
Step 1
Set up the Hyperledger Composer runtime with CouchDB
- Install the Hyperledger Composer development tools (as a non-root user)
- To install composer-cli, run the following command:
npm install -g composer-cli
- The composer-cli contains all the command-line operations for developing business networks.
- To install composer-rest-server, run the following command:
npm install -g composer-rest-server
The Hyperledger Composer Loopback Connector is used by the Hyperledger Composer REST server to generate the REST Explorer web page containing the REST APIs that have been generated for the model AFTER the connection to a business network, extraction of the models and schema for the network.
Only two tools are required for this tutorial. First of all, follow the instructions in Installing and developing with Hyperledger Composer to start the Hyperledger Fabric runtime.
Once the fabric runtime has started, you should see the four Docker services below via this command:
docker ps -a
View image at full size
A CouchDB image and CouchDB configuration need to be set up and enabled in order to use rich queries. Both the sets of optimizations have been done in the Hyperledger Composer v0.11+ with the Hyperledger Fabric V1 runtime.
Now, you can enable CouchDB for each peer or a specific peer, and also set the peer to depend on it. Once the CouchDB is enabled, a CouchDB instance is configured for the peer.
A docker-compose.yml file with the appropriate configuration looks something like this:
Step 2
Add a query to a business network
Queries are JSON objects defined in a query file (.qry) in the business network definition and the Hyperledger Composer Query language quite suitably follows the CouchDB Mango query language standards. These queries can be used to return the following resources:
- Assets
- Participants
- Historian data
We will be using some of the most simple queries defined in the Hyperledger Composer trade-network sample in this tutorial, which you can download from GitHub.
Every single query uses a simple syntax, defined by the initial query keyword, which is followed by the query name. For example, selectCommoditiesByOwner.
The description field must be having a human-readable and meaningful description of the query's function. The statement field contains the SELECT attribute, which further defines the registry or set of resources to query. Next it also contains the WHERE attribute, which defines the conditions that must be fulfilled to return resources.
Please note that the WHERE attribute can also contain semantic AND/OR modifiers. See the Hyperledger Composer Query Language documentation for more details regarding this.
To add a query
- Open an existing business network, and create a queries.qry file.
- In this file, you can describe multiple queries.
- I have shown an example query below. This query will return all commodities where the owner attribute matches the _$owner variable parameter.
When a transaction is submitted, the Historian feature in Hyperledger Composer maintains a history of ledger updates that have occurred in a business network.
The HistorianRecord is instantly updated and a registry of transactions is built up over time that includes the transaction inputs for a particular transaction as well. Along with this, the participants and identities involved in submitting those transactions are also registered into the historian record.
Two additional transactions are added to the queries.qry file to demonstrate the Historian feature in Hyperledger Composer:
Listing 1. Example query with additional transactions
query showCommodityAllHistorians{
description: "Select commodity all historians"
statement:
SELECT org.hyperledger.composer.system.HistorianRecord FROM
HistorianRegistry
WHERE (transactionType == 'AddAsset' OR transactionType ==
'UpdateAsset' OR transactionType == 'RemoveAsset')
}
query findCommmodityHistoriansWithTime{
description: "Find commodity historians after a specified time"
statement:
SELECT org.hyperledger.composer.system.HistorianRecord FROM
HistorianRegistry WHERE (transactionTimestamp > _$justnow)
}
Step 3: You can now cut and paste these two queries into the queries.qry file. DO NOT FORGET to save the file.
Deploy the business network to Hyperledger Fabric
The business network you now have should include the queries.qry file. Please deploy the business network with the following command if you have a local instance of Hyperledger Fabric:
composer network deploy -p hlfv1 -a ./trade-network.bna -i PeerAdmin -s adminpw
PLEASE NOTE: You should update the business network if you have already deployed the trade-network.bna, with this command:See the Hyperledger Composer Command Line documentation for more details.You can also choose to pass a full path to the trade-network.bna; DO KEEP IN MIND that the hlfv1 is the connection profile name to the Hyperledger Fabric runtime.
composer network update -p hlfv1 -a ./trade-network.bna -i PeerAdmin -s adminpw
If done the right way, a successful deployment message like this should show up:
View image at full size
The following command should show a chaincode container, once your business network has been deployed:
docker ps –a
View image at full size
Step 4
Expose rich queries via a REST API
Once your business network containing queries have been successfully deployed, launch the Composer REST server to expose the queries through a REST API.
A bespoke set of REST APIs based on your business network is generated once the Hyperledger Composer REST server is launched successfully.
- Begin with the following command from the command line with to start the REST server:
composer-rest-server
- Provide information as shown below:
View image at full size
- Open the REST server from a web browser by typing:
https://localhost:3000/explorer
You should see the query APIs below:
View image at full size
Your business network is currently very unlikely to have any data to query at this point.
You will need to create some participants, assets, and a Historian registry to test the queries in the next step.
Step 5
Create participants, assets, and a Historian registry
In this section, we will learn how to create two participants and two assets in the World State database and submit a trade transaction.
Create two participants
- Participants can be created from the REST server interface.
- Click Trader and then POST to create a Trader participant.
- The Trader participant requires three fields, as shown below. REPEAT THIS STEP TWICE to create two participants.
Listing 2. An example of a trader participant
{
"$class": "org.acme.trading.Trader",
"tradeId": "fenglian@email.com",
"firstName": "Fenglian",
"lastName": "Xu"
}
I will now teach you how to create two Commodity assets in the same way we created participants.Create two commodity assets
- Similar to the participants section, repeat this twice to successfully create two commodities.
Listing 3. An example of a commodity asset
{
"$class": "org.acme.trading.Commodity",
"tradingSymbol": "XYZ",
"description": "Soya",
"mainExchange": "Chicago",
"quantity": 50,
"owner": "dan@email.com"
}
- Listing 4. Two commodities created in the asset registryClick GET under the Commodity to show all commodities created.
[
{
"$class": "org.acme.trading.Commodity",
"tradingSymbol": "EMA",
"description": "Corn",
"mainExchange": "Euronext",
"quantity": 100,
"owner": "resource:org.acme.trading.Trader#dan@email.com"
},
{
"$class": "org.acme.trading.Commodity",
"tradingSymbol": "XYZ",
"description": "Soya",
"mainExchange": "Chicago",
"quantity": 50,
"owner": "resource:org.acme.trading.Trader#dan@email.com"
}
]
The id dan[at]email.com in the Participant registry has a URI prefix of resource:org.acme.trading.Trader# to the participant in question. This URI prefix is the one which the fully qualified relationship refers.Please keep in mind that the Fabric was modeled as such in the trade-network model that the owner attribute has a relationship to a Participant instance.
Create a trade transaction
If I am asked to explain a trade transaction, then I would put it as, a transaction defined to change the ownership of a Commodity. As mentioned earlier, all transactions are recorded in the Historian registry for this commodities-trading business network.
The historian registry maintains a history or record of transactions, their types, and the changes/deltas added by a particular transaction.
- Navigate to the Trade section and open POST.
- In the data field put the following data:
Listing 5. An example of a trade transaction
{
"$class": "org.acme.trading.Trade",
"commodity": "EMA",
"newOwner": "fenglian@email.com",
"transactionId": "",
"timestamp": "2017-08-07T15:04:33.790Z"
}
- This transaction changes the EMA commodity's owner rom dan to fenglian.Now simply click the “Try it out!” button to submit the trade transaction.
- Once this is done, you can verify this by going under the Commodity GET query clicking Try it out!
Listing 6. Updated EMA commodity owner
[
{
"$class": "org.acme.trading.Commodity",
"tradingSymbol": "EMA",
"description": "Corn",
"mainExchange": "Euronext",
"quantity": 100,
"owner": "resource:org.acme.trading.Trader#fenglian@email.com"
},
{
"$class": "org.acme.trading.Commodity",
"tradingSymbol": "XYZ",
"description": "Soya",
"mainExchange": "Chicago",
"quantity": 50,
"owner": "resource:org.acme.trading.Trader#dan@email.com"
}
]
Step 6
Query resources via the REST server
Now the next thing to be done is to learn how to use the rich Query functions and variable parameters to query business network data effectively via the REST server to get the best results.
Rich query data type support in Hyperledger Composer and REST server
The traditional Composer data types differ from the data types for parameters in queries exposed through the REST APIs due to the Hyperledger Composer Loopback Connector, as shown in Table 1.
Primitive data type mapping between Composer and the REST server
| Component | ||||
| Composer runtime | String | Double/Integer/Long | DateTime | Boolean |
| REST server | string | number | date | boolean |
The Composer runtime also supports these data types n addition to the above Composer runtime primitive data types:
- An associated relationship
This is always identified by a key property with a String type.
- A class containment relationship
The property data type is a class that can have another class containment relationship, etc. This is a nested relationship, and it is one of the primitive types.
- Enumerate: This is for a set of predefined data.
Query assets with parameters
Expand the query in the REST server UI. There is a list of queries defined in the queries.qry file earlier. You can use any of these to query the business network data now in the CouchDB database.
If you want to return commodities by the exchange symbol attribute then the selectComoditiesByExchange query allows you to do that.
View image at full size
To send the query, click Try it out! The results will be instantly displayed under the Try it out! Button, as shown below:
View image at full size
The Curl and the web browser URL for the query format with a parameter are shown in the response page.
The selectCommoditiesByOwner query demonstrates a query using the owner attribute. You can open the GET panel for the selectCommoditiesByOwner query and define the owner you wish to query:
View image at full size
The query result is shown below for the specified owner.
View image at full size
Query transaction Historian
As mentioned earlier, all transactions and asset updates are recorded in the Historian registry for this commodities-trading business network.
The HistorianRecord shows the changes to assets in a business network when a transaction is submitted and the participants and identities involved in submitting those transactions.
The Historian is defined in the Hyperledger Composer system name space, as shown in Listing 7.
See the Hyperledger Composer Historian Record documentation if you want to programmatically use the Historian.
Listing 7. Historian model
asset HistorianRecord identified by transactionId {
o String transactionId
o String transactionType
--> Transaction transactionInvoked
--> Participant participantInvoking optional
--> Identity identityUsed optional
o Event[] eventsEmitted optional
o DateTime transactionTimestamp
}
Navigate to the findCommodityHistoriansWithTime query, specify a date time value in UTC format, and click Try it out!
View image at full size
The listing 8 shows all Historian records of transactions after the specified date time in the Historian registry this query finds. Both system transactions and user-defined transactions are included in the transaction.
Listing 8. Historian records of transactions
[
{
"$class": "org.hyperledger.composer.system.HistorianRecord",
"transactionId": "0392a2e7-e056-4442-82c7-7d020cd0ee7a",
"transactionType": "AddParticipant",
"transactionInvoked": "resource:org.hyperledger.composer.system.Transaction#0392a2e7-e056-4442-82c7-7d020cd0ee7a",
"eventsEmitted": [],
"transactionTimestamp": "2017-08-07T15:08:19.047Z"
},
{
"$class": "org.hyperledger.composer.system.HistorianRecord",
"transactionId": "04f4410c-3470-4319-a6c5-3f5681d86488",
"transactionType": "AddAsset",
"transactionInvoked": "resource:org.hyperledger.composer.system.Transaction#04f4410c-3470-4319-a6c5-3f5681d86488",
"eventsEmitted": [],
"transactionTimestamp": "2017-08-07T15:13:56.223Z"
},
{
"$class": "org.hyperledger.composer.system.HistorianRecord",
"transactionId": "2529902c-1393-4537-98ca-f3e4d46a3164",
"transactionType": "Trade",
"transactionInvoked": "resource:org.hyperledger.composer.system.Transaction#2529902c-1393-4537-98ca-f3e4d46a3164",
"eventsEmitted": [
{
"$class": "org.acme.trading.TradeNotification",
"commodity": "resource:org.acme.trading.Commodity#EMA",
"eventId": "2529902c-1393-4537-98ca-f3e4d46a3164#0",
"timestamp": "2017-08-07T15:04:33.790Z"
}
],
"transactionTimestamp": "2017-08-07T15:04:33.790Z"
},
{
"$class": "org.hyperledger.composer.system.HistorianRecord",
"transactionId": "3f31add4-5e4d-40cd-84a5-e5553a14cb50",
"transactionType": "AddAsset",
"transactionInvoked": "resource:org.hyperledger.composer.system.Transaction#3f31add4-5e4d-40cd-84a5-e5553a14cb50",
"eventsEmitted": [],
"transactionTimestamp": "2017-08-07T15:13:03.584Z"
},
{
"$class": "org.hyperledger.composer.system.HistorianRecord",
"transactionId": "a50145ba-df31-4def-932e-cfc9707131ec",
"transactionType": "AddParticipant",
"transactionInvoked": "resource:org.hyperledger.composer.system.Transaction#a50145ba-df31-4def-932e-cfc9707131ec",
"eventsEmitted": [],
"transactionTimestamp": "2017-08-07T15:07:15.985Z"
}
]
Conclusion
From what I have heard and from what I have experienced myself, it becomes easy to perform complex and powerful queries on assets and transactions with the help of the rich Query language built into Hyperledger Composer.
The Hyperledger Composer Query language provides great flexibility for querying any attributes easily and without it, you would have to construct ad hoc queries in your code without this native Query language and then work to make sense of the results returned.
With the help of this step by step tutorial, I have attempted to show you:
- How to define queries in Hyperledger Composer
- How to deploy or update a business network to Hyperledger Fabric with CouchDB enabled
- How to set up a REST server and use that to query your business network.
Recommended steps (Additional)
Now that we are through with the major set up, here is a list of steps that I would recommend you, so you can strengthen your skills with the Hyperledger Composer.
- Check out the Developer tutorial for creating a Hyperledger Composer solution and experiment with rich queries in your own applications. This will guide you through creating a functioning application for an end-to-end blockchain solution and additionally help you try out rich queries for yourself in the future.
- Read this in-depth introduction to learn more about Hyperledger Composer.
- Get answers to your questions and contribute to the project by joining the Hyperledger Composer community.
- Create a decentralized energy network using Hyperledger Composer.
