Deploy Additional Function Apps
Before a new function app is deployed, check the following:
Make sure that the messages on the AWS SQS are increasing compared to the messages that are being consumed from SQS. To do so, in the Monitoring section of AWS SQS, check the Message Received stats against the Sent stats.
Make sure that the function app's SQS_FILES_READ_LIMIT environment variable is set to 200 by default, which means that the function reads 200 messages every 5 minutes. This can be increased based on your requirements. However, increasing this value beyond 1000 can lead to performance issues because the Queue Trigger function resides in the same function app.
If the number of messages to be processed from the AWS SQS has to be beyond 1000, do the following:
Disable the queue trigger app so that it doesn't starve the TimedSQS for resources.
Deploy additional trigger queue apps separately to handle the load on the Azure queue.
If private networking is used, then deploying additional function apps requires manually configuring the virtual network on each new app and adjusting the network configuration on the storage account. After you do this, then you need to restart the new function app. See Configure a Private Network.
Additional function apps are required as a result of the incoming event rate. If the number of messages being ingested is less than the incoming rate, then it is essential to add additional queue triggers. Either use the landing page to deploy additional queue triggers or use https://github.com/Azure/Azure-Sentinel/tree/master/Solutions/IllumioSaaS/Data%20Connectors for deploying.
Ingest Specific Network Traffic Logs
You can select the type of network traffic to be ingested. After you deploy the function app, go to function app, select environment variables and modify the value of the networkTrafficLogTypes key to ingest only the type of network traffic that you want.
The supported values are allowed, blocked, potentially_blocked, All, and unknown.
You can add any combinations of the supported values, separated by commas. For example, if you only want blocked and potentially blocked traffic to be ingested by the data connector, set the value to blocked, potentially_blocked.
