Docs
connector.conf
Specifications and example configurations for AWS connector.conf stanzas.
Specifications
[sqs_s3://<specify_name>]
host = <host-name>, Default is current system hostname
repo = <repository-name>
sourcetype = dataelicit/aws:aws-s3-accesslogs
frequency = <seconds> Interval to run the input
cron = <cron-expression>
Note: Cron has higher priority and will be considered for scheduling instead of frequency, if both are defined.
It is preferable to define only one, either cron or frequency.
sqs_queue_name = Name of SQS Queue
sqs_queue_region = AWS region of SQS Queue (Only 1 region, create another stanza for different region)
sqs_wait_time = <seconds> Wait time for messages to arrive in SQS
parse_csv_with_header = 0/1 Enable for CSV files
secret = <secret-stanza-name>
global = <global-stanza-name>
disabled = 0/1
[cloudtrail://<specify_name>]
host = <host-name>, Default is current system hostname
repo = <repository-name>
sourcetype = dataelicit/aws:aws-cloudtrail
frequency = <seconds> Interval to run the input
cron = <cron-expression>
Note: Cron has higher priority and will be considered for scheduling instead of frequency, if both are defined.
It is preferable to define only one, either cron or frequency.
sqs_queue_name = Name of SQS Queue
sqs_queue_region = AWS region of SQS Queue (Only 1 region, create another stanza for different region)
sqs_wait_time = <seconds> Wait time for messages to arrive in SQS
parse_csv_with_header = 0/1 Enable for CSV files
secret = <secret-stanza-name>
global = <global-stanza-name>
disabled = 0/1
[inspector://<specify_name>]
host = <host-name>, Default is current system hostname
repo = <repository-name>
sourcetype = dataelicit/aws:aws-inspector-findings
frequency = <seconds>
cron = <cron-expression>
Note: Cron has higher priority and will be considered for scheduling instead of frequency, if both are defined.
It is preferable to define only one, either cron or frequency.
regions = Comma seprated AWS regions without space Ex. ap-south-1,us-west-2
secret = <secret-stanza-name>
global = <global-stanza-name>
disabled = 0/1
[config://<specify_name>]
host = <host-name>, Default is current system hostname
repo = <repository-name>
sourcetype = dataelicit/aws:aws-config
frequency = <seconds>
cron = <cron-expression>
Note: Cron has higher priority and will be considered for scheduling instead of frequency, if both are defined.
It is preferable to define only one, either cron or frequency.
sqs_queue_name = Name of SQS Queue
sqs_queue_region = AWS region of SQS Queue (Only 1 region, create another stanza for different region)
sqs_wait_time = <seconds> Wait time for messages to arrive in SQS
secret = <secret-stanza-name>
global = <global-stanza-name>
disabled = 0/1
[metadata://<specify_name>]
host = <host-name>, Default is current system hostname
repo = <repository-name>
service = ec2
sourcetype = dataelicit/aws:aws-metadata
frequency = <seconds>
cron = <cron-expression>
Note: Cron has higher priority and will be considered for scheduling instead of frequency, if both are defined.
It is preferable to define only one, either cron or frequency.
regions = Comma seprated AWS regions without space Ex. ap-south-1,us-west-2
resource = ec2_volumes,ec2_instances,ec2_reserved_instances,ebs_snapshots,rds_instances,rds_reserved_instances,ec2_key_pairs,ec2_security_groups,ec2_images,ec2_addresses
Comma-seprated resources - Above are the valid resources, Use as per need
Specify "all" for all the resources
secret = <secret-stanza-name>
global = <global-stanza-name>
disabled = 0/1
[metadata://<specify_name>]
host = <host-name>, Default is current system hostname
repo = <repository-name>
service = s3
sourcetype = dataelicit/aws:aws-metadata
frequency = <seconds>
cron = <cron-expression>
Note: Cron has higher priority and will be considered for scheduling instead of frequency, if both are defined.
It is preferable to define only one, either cron or frequency.
regions = Comma seprated AWS regions without space Ex. ap-south-1,us-west-2
secret = <secret-stanza-name>
global = <global-stanza-name>
disabled = 0/1
[metadata://<specify_name>]
host = <host-name>, Default is current system hostname
repo = <repository-name>
service = vpc
sourcetype = dataelicit/aws:aws-metadata
frequency = <seconds>
cron = <cron-expression>
Note: Cron has higher priority and will be considered for scheduling instead of frequency, if both are defined.
It is preferable to define only one, either cron or frequency.
regions = Comma seprated AWS regions without space Ex. ap-south-1,us-west-2
resource = vpcs,vpc_network_acls,vpc_subnets
Comma-seprated resources - Above are the valid resources, Use as per need
Specify "all" for all the resources
secret = <secret-stanza-name>
global = <global-stanza-name>
disabled = 0/1
[metadata://<specify_name>]
host = <host-name>, Default is current system hostname
repo = <repository-name>
service = elb
sourcetype = dataelicit/aws:aws-metadata
frequency = <seconds>
cron = <cron-expression>
Note: Cron has higher priority and will be considered for scheduling instead of frequency, if both are defined.
It is preferable to define only one, either cron or frequency.
regions = Comma seprated AWS regions without space Ex. ap-south-1,us-west-2
resource = classic_load_balancers,application_load_balancers
Comma-seprated resources - Above are the valid resources, Use as per need
Specify "all" for all the resources
secret = <secret-stanza-name>
global = <global-stanza-name>
disabled = 0/1
[metadata://<specify_name>]
host = <host-name>, Default is current system hostname
repo = <repository-name>
service = eks
sourcetype = dataelicit/aws:aws-metadata
frequency = <seconds>
cron = <cron-expression>
Note: Cron has higher priority and will be considered for scheduling instead of frequency, if both are defined.
It is preferable to define only one, either cron or frequency.
regions = Comma seprated AWS regions without space Ex. ap-south-1,us-west-2
resource = eks_describe_clusters,eks_list_nodegroups,eks_describe_nodegroups,eks_describe_update,eks_list_tags_for_resource,eks_list_addon,eks_describe_addon,eks_describe_fargate_profile,eks_describe_identity_provider_config,eks_describe_addon_versions
Comma-seprated resources - Above are the valid resources, Use as per need
Specify "all" for all the resources
secret = <secret-stanza-name>
global = <global-stanza-name>
disabled = 0/1
[metadata://<specify_name>]
host = <host-name>, Default is current system hostname
repo = <repository-name>
service = iam
sourcetype = dataelicit/aws:aws-metadata
frequency = <seconds>
cron = <cron-expression>
Note: Cron has higher priority and will be considered for scheduling instead of frequency, if both are defined.
It is preferable to define only one, either cron or frequency.
regions = Comma seprated AWS regions without space Ex. ap-south-1,us-west-2
resource = iam_users,iam_list_policy,iam_list_policy_local_and_only_attached,iam_server_certificates,iam_list_role_policies,iam_list_mfa_devices,iam_list_signing_certificates,iam_list_ssh_public_keys
Comma-seprated resources - Above are the valid resources, Use as per need
Specify "all" for all the resources
secret = <secret-stanza-name>
global = <global-stanza-name>
disabled = 0/1
[metadata://<specify_name>]
host = <host-name>, Default is current system hostname
repo = <repository-name>
service = network-firewall
sourcetype = dataelicit/aws:aws-metadata
frequency = <seconds>
cron = <cron-expression>
Note: Cron has higher priority and will be considered for scheduling instead of frequency, if both are defined.
It is preferable to define only one, either cron or frequency.
regions = Comma seprated AWS regions without space Ex. ap-south-1,us-west-2
resource = network_firewall_describe_firewalls,network_firewall_describe_logging_configurations,network_firewall_describe_firewall_policies,network_firewall_describe_rule_groups,network_firewall_list_tags_for_resource,network_firewall_describe_resource_policies
Comma-seprated resources - Above are the valid resources, Use as per need
Specify "all" for all the resources
secret = <secret-stanza-name>
global = <global-stanza-name>
disabled = 0/1
[cloudwatch://<specify_name>]
host = <host-name>, Default is current system hostname
repo = <repository-name>
service = ec2
sourcetype = dataelicit/aws:aws-cloudwatch
frequency = <seconds>
cron = <cron-expression>
Note: Cron has higher priority and will be considered for scheduling instead of frequency, if both are defined.
It is preferable to define only one, either cron or frequency.
regions = Comma seprated AWS regions without space Ex. ap-south-1,us-west-2
AutoScalingGroupName = <Comma-sperated list of ASG ARNs>
Specify "all" for all the ASGs
InstanceId = <Comma-sperated list of instance Ids> Ex. i-0526b4e1cf,i-09a7d05a4fd
Specify "all" for all the instances present
InstanceType = <Comma-sperated list of Instance Types>
Specify "all" for all the instances present
ImageId = <Comma-sperated list of Image Ids>
Specify "all" for all the instances present
statistics = Average,Sum,SampleCount,Maximum,Minimum
Comma-seprated statistics - Above are the valid statistics, Use as per need
Specify "all" for all the statistics
metrics = CPUCreditBalance,CPUCreditUsage,CPUUtilization,DiskReadOps,DiskWriteOps,DiskReadBytes,DiskWriteBytes,NetworkIn,NetworkOut,NetworkPacketsIn,NetworkPacketsOut,StatusCheckFailed,StatusCheckFailed_Instance,StatusCheckFailed_System,MetadataNoToken,CPUCreditUsage,CPUCreditBalance,CPUSurplusCreditBalance,CPUSurplusCreditsCharged,EBSReadOps,EBSWriteOps,EBSReadBytes,EBSWriteBytes,EBSIOBalance%,EBSByteBalance%
Comma-seprated metrics - Above are the valid metrics, Use as per need
Specify "all" for all the metrics
query_window_size = <Lookback time in seconds> Ex. 7200
period = <seconds> The granularity, in seconds, of the returned data points. Can be as short as 60 seconds, and must be in multiple of 60. Default: 300
secret = <secret-stanza-name>
global = <global-stanza-name>
disabled = 0/1
[cloudwatch://<specify_name>]
host = <host-name>, Default is current system hostname
repo = <repository-name>
service = ebs
sourcetype = dataelicit/aws:aws-cloudwatch
frequency = <seconds>
cron = <cron-expression>
Note: Cron has higher priority and will be considered for scheduling instead of frequency, if both are defined.
It is preferable to define only one, either cron or frequency.
regions = Comma seprated AWS regions without space Ex. ap-south-1,us-west-2
VolumeId = <Comma-sperated list of Volume Ids>
Specify "all" for all the VolumeIds present
statistics = Average,Sum,SampleCount,Maximum,Minimum
Comma-seprated statistics - Above are the valid statistics, Use as per need
Specify "all" for all the statistics
metrics = VolumeReadBytes,VolumeWriteBytes,VolumeReadOps,VolumeWriteOps,VolumeTotalReadTime,VolumeTotalWriteTime,VolumeIdleTime,VolumeQueueLength,VolumeThroughputPercentage,VolumeConsumedReadWriteOps,BurstBalance
Comma-seprated metrics - Above are the valid metrics, Use as per need
Specify "all" for all the metrics
query_window_size = <Lookback time in seconds> Ex. 7200
period = <seconds> The granularity, in seconds, of the returned data points. Can be as short as 60 seconds, and must be in multiple of 60. Default: 300
secret = <secret-stanza-name>
global = <global-stanza-name>
disabled = 0/1
[cloudwatch://<specify_name>]
host = <host-name>, Default is current system hostname
repo = <repository-name>
service = elb
sourcetype = dataelicit/aws:aws-cloudwatch
frequency = <seconds>
cron = <cron-expression>
Note: Cron has higher priority and will be considered for scheduling instead of frequency, if both are defined.
It is preferable to define only one, either cron or frequency.
regions = Comma seprated AWS regions without space Ex. ap-south-1,us-west-2
LoadBalancer = <Comma-sperated list of Load Balancer ARNs>
Specify "all" for all the Load Balancers present
ELBtype = application | network
statistics = Average,Sum,SampleCount,Maximum,Minimum
Comma-seprated statistics - Above are the valid statistics, Use as per need
Specify "all" for all the statistics
metrics = <application metrics> RequestCount,RequestCountPerTarget,ActiveConnectionCount,ProcessedBytes,TargetConnectionErrorCount,TargetResponseTime,TargetTLSNegotiationErrorCount,HTTPCode_Target_2XX_Count,HTTPCode_Target_3XX_Count,HTTPCode_Target_4XX_Count,HTTPCode_Target_5XX_Count,HTTPCode_ELB_4XX_Count,HTTPCode_ELB_5XX_Count,ClientTLSNegotiationErrorCount,ConsumedLCUs,IPv6ProcessedBytes,IPv6RequestCount,HealthyHostCount,UnHealthyHostCount,NewConnectionCount,RejectedConnectionCount,RuleEvaluations
<network metrics> ActiveFlowCount,ConsumedLCUs,HealthyHostCount,NewFlowCount,ProcessedBytes,TCP_Client_Reset_Count,TCP_ELB_Reset_Count,TCP_Target_Reset_Count,UnHealthyHostCount
Comma-seprated metrics - Above are the valid metrics, Use as per need
Specify "all" for all the metrics
query_window_size = <Lookback time in seconds> Ex. 7200
period = <seconds> The granularity, in seconds, of the returned data points. Can be as short as 60 seconds, and must be in multiple of 60. Default: 300
secret = <secret-stanza-name>
global = <global-stanza-name>
disabled = 0/1
[cloudwatch://<specify_name>]
host = <host-name>, Default is current system hostname
repo = <repository-name>
service = lambda
sourcetype = dataelicit/aws:aws-cloudwatch
frequency = <seconds>
cron = <cron-expression>
Note: Cron has higher priority and will be considered for scheduling instead of frequency, if both are defined.
It is preferable to define only one, either cron or frequency.
regions = Comma seprated AWS regions without space Ex. ap-south-1,us-west-2
FunctionName = <Comma-sperated list of Lambda Function names>
Specify "all" for all the functions present
statistics = Average,Sum,SampleCount,Maximum,Minimum
Comma-seprated statistics - Above are the valid statistics, Use as per need
Specify "all" for all the statistics
metrics = Invocations,Errors,'Dead Letter Error',Duration,Throttles,IteratorAge,ConcurrentExecutions,UnreservedConcurrentExecutions
Comma-seprated metrics - Above are the valid metrics, Use as per need
Specify "all" for all the metrics
query_window_size = <Lookback time in seconds> Ex. 7200
period = <seconds> The granularity, in seconds, of the returned data points. Can be as short as 60 seconds, and must be in multiple of 60. Default: 300
secret = <secret-stanza-name>
global = <global-stanza-name>
disabled = 0/1
[cloudwatch://<specify_name>]
host = <host-name>, Default is current system hostname
repo = <repository-name>
service = s3
sourcetype = dataelicit/aws:aws-cloudwatch
frequency = <seconds>
cron = <cron-expression>
Note: Cron has higher priority and will be considered for scheduling instead of frequency, if both are defined.
It is preferable to define only one, either cron or frequency.
regions = Comma seprated AWS regions without space Ex. ap-south-1,us-west-2
BucketName = <Comma-sperated list of S3 Bucket names>
Specify "all" for all the buckets present
StorageType = StandardStorage,StandardIAStorage,OneZoneIAStorage,ReducedRedundancyStorage,GlacierStorage,AllStorageTypes
Comma-seprated Storage Types - Above are the valid types, Use as per need
Specify both BucketName & StorageType for advance filtering
statistics = Average,Sum,SampleCount,Maximum,Minimum
Comma-seprated statistics - Above are the valid statistics, Use as per need
Specify "all" for all the statistics
metrics = BucketSizeBytes,NumberOfObjects
Comma-seprated metrics - Above are the valid metrics, Use as per need
Specify "all" for all the metrics
query_window_size = <Lookback time in seconds> Ex. 7200
period = <seconds> The granularity, in seconds, of the returned data points. Can be as short as 60 seconds, and must be in multiple of 60. Default: 300
secret = <secret-stanza-name>
global = <global-stanza-name>
disabled = 0/1Example
Example connector.conf for collecting CloudWatch, CloudTrail, S3 access logs, Config, Inspector, and AWS metadata.
[cloudwatch://ec2] sourcetype = dataelicit/aws:aws-cloudwatch frequency = 300 regions = ap-south-1,us-west-2 service = ec2 secret = aws_creds global = aws AutoScalingGroupName = all InstanceId = i-0x101x101x101x,i-01x01x01x01x10x statistics = Average,SampleCount,Sum query_window_size = 7200 [cloudwatch://ebs] sourcetype = dataelicit/aws:aws-cloudwatch cron = 0 0 1 * * regions = ap-south-1,us-west-2 service = ebs secret = aws_creds global = aws VolumeId = all statistics = all query_window_size = 7200 [cloudwatch://elb] sourcetype = dataelicit/aws:aws-cloudwatch frequency = 300 regions = ap-south-1,us-west-2 service = elb secret = aws_creds global = aws ELBtype = application LoadBalancer = all statistics = all query_window_size = 7200 [cloudwatch://lambda] sourcetype = dataelicit/aws:aws-cloudwatch cron = 0 0 1 * * regions = ap-south-1,us-west-2 service = lambda secret = aws_creds global = aws FunctionName = all statistics = all query_window_size = 7200 [cloudwatch://s3] sourcetype = dataelicit/aws:aws-cloudwatch frequency = 300 regions = ap-south-1,us-west-2 service = s3 secret = aws_creds global = aws BucketName = all query_window_size = 7200 [cloudtrail://cloudtrail] source = cloudtrail sourcetype = dataelicit/aws:aws-cloudtrail frequency = 86400 sqs_queue_name = logconnectorqueue sqs_queue_region = ap-south-1 sqs_wait_time = 10 parse_csv_with_header = 0 secret = aws_creds global = aws [sqs_s3://access_logs] source = access_logs sourcetype = dataelicit/aws:aws-s3-accesslogs frequency = 86400 assume_role_arn = 'arn:aws:iam::101010101010:role/Admin' sqs_queue_name = logconnectorqueue sqs_queue_region = ap-south-1 sqs_wait_time = 10 parse_csv_with_header = 1 secret = aws_creds global = aws [inspector://findings] sourcetype = dataelicit/aws:aws-inspector-findings frequency = 86400 regions = ap-south-1,us-west-2 secret = aws_creds global = aws [config://testconfig] sourcetype = dataelicit/aws:aws-config frequency = 86400 sqs_queue_name = logqueue sqs_queue_region = us-west-2 sqs_wait_time = 10 secret = aws_creds global = aws [metadata://ec2] sourcetype = dataelicit/aws:aws-metadata cron = 0 0 1 * * regions = ap-south-1,us-west-2 service = ec2 resource = all secret = aws_creds global = aws [metadata://s3] sourcetype = dataelicit/aws:aws-metadata cron = 0 0 1 * * regions = ap-south-1,us-west-2 service = s3 period = 300 secret = aws_creds global = aws [metadata://vpc] sourcetype = dataelicit/aws:aws-metadata cron = 0 0 1 * * regions = ap-south-1,us-west-2 service = vpc resource = all secret = aws_creds global = aws [metadata://elb] sourcetype = dataelicit/aws:aws-metadata frequency = 86400 regions = ap-south-1,us-west-2 service = elb resource = application_load_balancers secret = aws_creds global = aws [metadata://eks] sourcetype = dataelicit/aws:aws-metadata frequency = 86400 regions = ap-south-1,us-west-2 service = eks resource = eks_describe_clusters,eks_list_nodegroups,eks_describe_nodegroups,eks_describe_update,eks_list_tags_for_resource secret = aws_creds global = aws [metadata://iam] sourcetype = dataelicit/aws:aws-metadata frequency = 86400 regions = ap-south-1,us-west-2 service = iam resource = iam_users,iam_server_certificates,iam_list_mfa_devices,iam_list_signing_certificates,iam_list_ssh_public_keys secret = aws_creds global = aws [metadata://networkfirewall] sourcetype = dataelicit/aws:aws-metadata frequency = 86400 regions = ap-south-1,us-west-2 service = network-firewall resource = all period = 300 secret = aws_creds global = aws
Note
Use either cron or frequency. cron takes priority when both are set.
