Author: Kushagra Kakkar & Rahul Kumar
Introduction:
This tutorial will help you in understanding the concepts of the Apisero Generic Logging Framework and how to use it to perform logging onto different Logging platforms such as Splunk, ELK, etc.
What is Apisero Generic Logger, and when to use it?
Apisero Generic Logger is a framework which is used instead of the Logger component provided by Mulesoft to enhance the logging capabilities and provides features such as Masking, Logging into Database, etc.
We can use Generic Logger in case we have to send the logs to different platforms, or we need some enriched logs instead of the default ones provided by Mulesoft.
Prerequisites:
1. The first and foremost prerequisite is that the person should be a part of Apisero in order to use the generic logger or have access to Apisero’s organization on Anypoint Platform.
2. After you have access, go to: https://anypoint.mulesoft.com/exchange/4c2825dd-e397-4b98-8eb5-fdf4c967c16e/apisero-generic-logging-connector/minor/2.0/ and download the latest version of logger framework:
3. Maven should be installed on the computer in order to build the connector
4. Make sure that the Mule Runtime Version in your Anypoint Studio is 4.3.0; otherwise MUnit will fail for your flows.
Tutorial:
We’ll use the Generic Logger to log onto Splunk as an example for this tutorial. So, first, we should create a free account on Splunk by visiting https://www.splunk.com/.
- After the account is created, we need to set up Splunk in order to get the token and also to set on which index the logging would occur. We’ll use the main index on Splunk for logging. The below steps need to be followed:
a) Login to the Splunk dashboard with your credentials received on registered mail:
b) Now go to Settings -> Data Inputs:
c) Now, we’ll create an HTTP Collector to collect the logs from our Generic logger:
d) Provide a name for your Collector
e) Now select main index, then click on Review and Submit
f) This will display the token value on your screen for the particular HTTP Request Collector. Remember to store this token somewhere. Even in case you lose the token, it can be easily viewed from Settings->Data Inputs-> HTTP Event Collector.
- Now that we have configured Splunk, our next step is to create the Generic Logger Connector.
- Add the following in your Maven settings.xml under <servers>:
<server>
<id>MuleRepository</id>
<username>Apisero Nexus Repo</username>
<password>Apisero Nexus Pass</password>
</server>
- Add the following in your Maven settings.xml under <profiles>:
<profile>
<id>Mule</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<repositories>
<repository>
<id>MuleRepository</id> <name>MuleRepository</name>
<url>https://repository.mulesoft.org/nexus-ee/content/repositories/releases-ee/</url>
<layout>default</layout>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
<repository>
<id>mulesoft-releases</id>
<name>MuleSoft Releases Repository</name>
<url>https://repository.mulesoft.org/releases/</url>
<layout>default</layout>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>mulesoft-release</id>
<name>mulesoft release repository</name>
<layout>default</layout>
<url>https://repository.mulesoft.org/releases/</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
- Extract the downloaded Generic Logger Framework zip file, which was provided in the Prerequisites. You’ll see the below contents:
- Go to the <Your-Directory>\apisero-generic-logging-connector-2.0.1-custom\logging-connector\apisero-generic-logging-connector\src\main\resources\org\mule\extensions\smart\connector and open module-logging.xml
- Now in this module-logging.xml, Comment the flows which are not required under the For Each Target; for our case we will only uncomment Splunk and comment all the other flows under For Each Target
- Now go back to the connector folder where the pom.xml file is located <Your-Directory>\apisero-generic-logging-connector-2.0.1-custom\logging-connector-2.0.1\apisero-generic-logging-connector.
- Open Cmd at this location and run the following command:
mvn clean install -DskipTests
- In case the build is successful, you’ll get the below prompt:
- Open Anypoint Studio and Remember to use Mule Runtime Version 4.3.0.
- Now import this connector into your Project via Manage Modules. Then add the connector as a Maven Dependency.
- The connector would be present in <Your-Directory>\apisero-generic-logging-connector-2.0.1-custom\logging-connector-2.0.1\apisero-generic-logging-connector\target as a jar file.
- If all the above steps have been completed correctly, you will see a connector named LoggingConnector in your Mule Palette
- After this, Add the Mule Secure Configuration Property Module from Exchange into your Project.
- We’ll use the below flow for our application; we have used default Listener Config and have used Log Entry and Log Exit components from Logging connector:
- Now we need to create two files under src/main/resources: properties/dev.yaml and properties/dev-secure.yaml
- Now provide the below properties in dev.yaml:
maskValues: “<Name of Payload element to be masked, in our case we are taking DOB>”
splunkConfig:
protocol: “HTTPS”
host: “<Your Splunk URL>”
port: “8088”
basepath: “/services/collector”
retryCount: “3”
retryIntervalInMilliseconds: “60000”
loggingFramework:
logTo:
info: “splunk,default”
debug: “splunk,default”
warn: “splunk,default”
fatal: “splunk,default”
error: “splunk,default”
“trace”: “splunk,default”
Note: maskValues is a mandatory field; it can be kept as an empty string in case no values are required to be masked, but providing it in the yaml file is necessary.
- Now for the dev-secure.yaml, We need to encrypt the Authorization Token of your Splunk HTTP Event Collector. We’ll use https://secure-properties-api.us-e1.cloudhub.io/ utility to encrypt our token using Blowfish Algorithm. We are using abcd as our key.
- Now provide this encrypted value in the dev-secure.yaml as follows:
splunkConfig:
authToken: “![<Your Encrypted Token here>]”
- Create a global.xml and create Configuration Properties and Secure Property Config in the Global elements as given in the below screenshots:
- Now Add the below Configurations in your Configuration XML of global.xml for sending logs to our Splunk dashboard:
<http:request-config name=“Splunk_HTTP_Config” doc:name=“HTTP Request configuration”
doc:id=“01a97145-ae01-4f03-9f36-cef1da9f2e97” >
<http:request-connection protocol=“${splunkConfig.protocol}”
host=“${splunkConfig.host}” port=“${splunkConfig.port}” >
<tls:context >
<tls:trust-store insecure=“true” />
</tls:context>
</http:request-connection>
</http:request-config>
- We can run our application and check on Splunk whether we are receiving any logs.
- After our application has been executed, go to Splunk Dashboard-> Apps-> Search & Reporting and search for “*”. This is a wildcard and will display all the logs for the provided time period:
As you can see in the above Screenshot, we are getting the Logs for both Entry and Exit according to our Flow, and as we provided DOB in maskValues in the dev.yaml file, it is evident from the above that it is successfully masked.
This completes our tutorial for the Apisero Generic Logger.
