Deploy germainAPM Services (on-premise)

Installation

Extract the contents of the Germain APM Service distribution to folder (ex: /opt/germainapm) and set the APM_HOME environment variable to that location.

License deployment

Please upload your license key to the SERVICES_HOME/var/licenses folder. You can upload it to a different location but afterwards you need to update SERVICES_HOME/config/common.properties accordingly:

Configuration

All of the germain APM services are available in GermainAPMService distribution file and can be configured via yml files in the SERVICES_HOME/config folder.

Database configuration

To connect germain APM Services to your germain APM databases, edit SERVICES_HOME/config/common.properties:

Please contact us if you need the latest version

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 # Shared Germain APM properties. # License key location germain.licenseDirectory=var/licenses # Datamart connection germain.datamart.maximumPoolSize=20 germain.datamart.minimumIdle=0 germain.datamart.poolName=DatamartPool # MySQL germain.datamart.classname=com.mysql.jdbc.Driver germain.datamart.url=jdbc:mysql://localhost:3306/APM_DATAMART?rewriteBatchedStatements=true&useSSL=false germain.datamart.username=apm_datamart germain.datamart.password=apm_datamart # Oracle #germain.datamart.classname=oracle.jdbc.driver.OracleDriver #germain.datamart.url=jdbc:oracle:thin:@localhost:1521:ORCL #germain.datamart.username=APM_DATAMART #germain.datamart.password=APM_DATAMART #germain.datamart.properties.version=12 #germain.datamart.properties.parallelism=4 #germain.datamart.properties.useBoundTimestamps=false # Config connection germain.config.maximumPoolSize=10 germain.config.minimumIdle=0 germain.config.poolName=ConfigPool # MySQL germain.config.classname=com.mysql.jdbc.Driver germain.config.url=jdbc:mysql://localhost:3306/APM_CONFIG?useSSL=false germain.config.username=apm_config germain.config.password=apm_config # Oracle #germain.config.classname=oracle.jdbc.driver.OracleDriver #germain.config.url=jdbc:oracle:thin:@localhost:1521:ORCL #germain.config.username=APM_CONFIG #germain.config.password=APM_CONFIG # ActiveMQ Broker germain.messaging.brokerUrl=failover:(tcp://localhost:61616?daemon=true&keepAlive=true) # Number of consumers listening on each queue. Can either be a upper-lower # limit (ex: 1-10) or just an upper limit (ex: 10) germain.messaging.listenerCount=2 # Max buffer size for kryo serialization (in mb) germain.messaging.maxBufferSize=64 # Queue names germain.messaging.queues.storageDefaultQueue=apm.storage germain.messaging.queues.storageAnalyticsQueue=apm.storage.analytics germain.messaging.queues.analyticsQueue=apm.analytics germain.messaging.queues.actionQueue=apm.action germain.messaging.queues.actionControlQueue=apm.action.control germain.messaging.queues.sessionQueue=apm.session germain.messaging.queues.rtmQueue=apm.rtm germain.messaging.queues.dimensionIndexQueue=apm.dimensionIndex germain.messaging.queues.contentIndexQueue=apm.contentIndex germain.messaging.topics.configTopic=apm.config # Mail settings spring.mail.from=alert@germainapm.com spring.mail.host=mail.germainapm.com spring.mail.username=alert@germainapm.com spring.mail.password=password spring.mail.port=587 spring.mail.properties.mail.smtp.auth=true spring.mail.properties.mail.smtp.starttls.enable=true

Service installation

Windows

To install a service, first go to SERVICES_HOME/bin/WinSW and run the .bat file for the version of .net you have. If you do not know, there are instruction for checking here: https://docs.microsoft.com/en-us/dotnet/framework/migration-guide/how-to-determine-which-versions-are-installed.

Choose the highest version applicable to your server:

  • set-dotnet-2.bat for versions below 4

  • set-dotnet-4.bat for versions 4 to 4.6

  • set-dotnet-461.bat for versions 4.61 or greater

Now go to SERVICES_HOME/bin folder and run the following command (requires administrative privileges):

1 <service name>.exe install

You can adjust the Java runtime properties (eg. memory) for the services by editing the corresponding *Services.xml files in the bin directory.

Service uninstallation

Windows

To uninstall a service, go to SERVICES_HOME/bin folder and run the following command (requires administrative privileges):

1 <service name>.exe uninstall

Service configuration

The configuration files can be found in the SERVICES_HOME/config folder.

Please contact us if you need the latest version

Action (action-services.yml)

1 2 3 4 action: name: Germain APM Action Service server: port: 8095

Aggregation (aggregator-services.yml)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 service: name: AggregatorService server: port: 8092 aggregation: dateTimeFormat: yyyy-MM-dd HH:mm:ss maxConcurrentJobs: 8 bufferDirectory: var/buffers bufferDailyAggJobs: true schedule: one_minute: 5 * * * * * five_minutes: 15 */5 * * * * sixty_minutes: 25 30 * * * * # Aggregation from hourly to daily. Runs once after midnight and then once again # to include any data that was backfilled (needs to be kept in sync with # lookBehindMinutes of backfill). one_day: 30 45 0,12 * * * purge: 0 50 * * * * # Ingestion delay determines how quickly raw data is rolled up into # aggregates. This needs to be kept in sync with the default aggregation # plan specified under germain.apm.monitoringConfig.systemConfig.datamart! ingestionDelay: one_minute: 5 five_minutes: 10 # Exclude facts types from daily aggregation job # dailyExclusions: # - SiebelOutboundHttpRequest batchPurge: # Controls purge of Spring Batch tables enabled: true # Individual queries to run queries: # MySQL version - delete from BATCH_STEP_EXECUTION_CONTEXT where STEP_EXECUTION_ID in (select STEP_EXECUTION_ID from BATCH_STEP_EXECUTION where START_TIME < ?) limit ? - delete from BATCH_STEP_EXECUTION where START_TIME < ? limit ? - delete from BATCH_JOB_EXECUTION_CONTEXT where JOB_EXECUTION_ID in (select JOB_EXECUTION_ID from BATCH_JOB_EXECUTION where START_TIME < ?) limit ? - delete from BATCH_JOB_EXECUTION_PARAMS where JOB_EXECUTION_ID in (select JOB_EXECUTION_ID from BATCH_JOB_EXECUTION where START_TIME < ?) limit ? - delete from BATCH_JOB_EXECUTION where START_TIME < ? limit ? # Oracle version # - delete from BATCH_STEP_EXECUTION_CONTEXT where STEP_EXECUTION_ID in (select STEP_EXECUTION_ID from BATCH_STEP_EXECUTION where START_TIME < ?) and rownum <= ? # - delete from BATCH_STEP_EXECUTION where START_TIME < ? and rownum <= ? # - delete from BATCH_JOB_EXECUTION_CONTEXT where JOB_EXECUTION_ID in (select JOB_EXECUTION_ID from BATCH_JOB_EXECUTION where START_TIME < ?) and rownum <= ? # - delete from BATCH_JOB_EXECUTION_PARAMS where JOB_EXECUTION_ID in (select JOB_EXECUTION_ID from BATCH_JOB_EXECUTION where START_TIME < ?) and rownum <= ? # - delete from BATCH_JOB_EXECUTION where START_TIME < ? and rownum <= ? backfillJobs: # Backfill to account for latency of session tracking - name: session-segment-backfill schedule: 50 * * * * * lookBehindMinutes: 10 spanMinutes: 1 factTypes: - UxSessionSegment - UxSession # Backfill job for generic data runs every 15 minutes looking back 3 hours - name: generic-backfill-3h schedule: 0 0/15 * * * * lookBehindMinutes: 180 spanMinutes: 15 factTypes: - GenericEvent - GenericMetric - GenericTransaction - MobileEvent - MobileMetric - MobileTransaction - UxSession - GenericBusinessProcess - GenericBusinessProcessStep # Backfill job for generic data runs every 60 minutes looking back 12 hours - name: generic-backfill-12h schedule: 0 7 * * * * lookBehindMinutes: 720 spanMinutes: 60 factTypes: - GenericEvent - GenericMetric - GenericTransaction - MobileEvent - MobileMetric - MobileTransaction - UxSession - GenericBusinessProcess - GenericBusinessProcessStep # Backfill for Siebel log data runs every 15 minutes looking back 3 hours - name: siebel-backfill-3h schedule: 0 0/15 * * * * lookBehindMinutes: 180 spanMinutes: 15 factTypes: - SiebelDatabaseTransaction - SiebelErrorEvent - SiebelServiceTransaction - SiebelStateServiceTransaction - SiebelTaskServiceTransaction - SiebelToolkitServiceTransaction - SiebelUserTransaction - SiebelWebEngineTransaction - SiebelWebServiceTransaction - SiebelWorkflowServiceTransaction - SiebelWorkflowStepTransaction - SiebelWorkflowTransaction - SiebelBusinessProcess - SiebelBusinessProcessStep # Backfill for Siebel log data runs every 60 minutes looking back 12 hours - name: siebel-backfill-12h schedule: 0 7 * * * * lookBehindMinutes: 720 spanMinutes: 60 factTypes: - SiebelDatabaseTransaction - SiebelErrorEvent - SiebelServiceTransaction - SiebelStateServiceTransaction - SiebelTaskServiceTransaction - SiebelToolkitServiceTransaction - SiebelUserTransaction - SiebelWebEngineTransaction - SiebelWebServiceTransaction - SiebelWorkflowServiceTransaction - SiebelWorkflowStepTransaction - SiebelWorkflowTransaction - SiebelBusinessProcess - SiebelBusinessProcessStep spring.batch.job.enabled: false

Analytics (analytics-services.yml)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 # General service properties analytics: name: Germain APM Analytics Services # Directory that will hold data buffers bufferDirectory: var/buffers # If true, will delete buffer files after processing bufferPurge: true # Schedule for BP cleanup job (every 15 minutes) cleanupSchedule: "0 15 * * * *" executor: # Should be at least 2x number of BPs coreSize: 32 maxSize: 64 # Should match expected number of concurrent user sessions queueSize: 2000 # REST server properties server: port: 8094

Session Tracking (session-tracking.yml)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 # REST server properties server: port: 8093 # General service properties session: name: Germain APM Session Tracking # Directory that will hold data buffers for sessions bufferDirectory: var/buffers # If true, will delete buffer files after processing bufferPurge: true # Session timeout interval (seconds) timeoutInterval: 3600 # Latency of session data (default=120 seconds) maxLatency: 120 # Min session segment duration (seconds) segmentDuration: 30 executor: coreSize: 32 maxSize: 64 queueSize: 4000

Storage (storage-services.yml)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 # REST server properties server: port: 8091 # General service properties storage: name: Germain APM Storage Services # Nightly at 00:15 am rawPurgeSchedule: "0 15 0 * * *" # Nightly at 01:15 am aggregatePurgeSchedule: "0 15 1 * * *" # Number of concurrent threads threadPoolSize: 64 # Enables queue / consumer availability monitoring. queueMonitoringEnabled: true # Number of minutes after which to consider a queue unavailable. queueAvailabilityThreshold: 5 # Search engine indexer indexer: # NoOp vendor: NOOP # Solr Adapter #vendor: SOLR #url: http://localhost:8983/solr/ # ES # vendor: ELASTIC_SEARCH # url: http://localhost:9200/ # main index name dimensionIndexName: apm contentIndexName: content # Index refresh schedule (every 1h) refreshSchedule: 0 0 * * * * # Disables purge and maintenance of raw fact tables purgeExclusions: # - GenericMetric # Disables relay to analytics services analyticsExclusions: - com.germainsoftware.apm.data.model.UxSession - com.germainsoftware.apm.data.model.UxSessionSegment - com.germainsoftware.apm.data.model.GenericBusinessProcess - com.germainsoftware.apm.data.model.GenericBusinessProcessStep - com.germainsoftware.apm.data.model.SiebelBusinessProcess - com.germainsoftware.apm.data.model.SiebelBusinessProcessStep - com.germainsoftware.apm.data.model.SiebelDatabaseTransaction - com.germainsoftware.apm.data.model.SiebelWorkflowStepTransaction - com.germainsoftware.apm.data.model.SiebelServiceTransaction - com.germainsoftware.apm.data.model.SiebelToolkitServiceTransaction - com.germainsoftware.apm.data.model.SiebelStateServiceTransaction - com.germainsoftware.apm.data.model.SiebelTaskServiceTransaction - com.germainsoftware.apm.data.model.SiebelWorkflowServiceTransaction # Disables timeshift adjustment for facts that are generated by services timeshiftExclusions: - com.germainsoftware.apm.data.model.GenericBusinessProcess - com.germainsoftware.apm.data.model.GenericBusinessProcessStep - com.germainsoftware.apm.data.model.SiebelBusinessProcess - com.germainsoftware.apm.data.model.SiebelBusinessProcessStep dimensionCache: indexEnabled: false # How frequently to update dimension index in milliseconds. indexInterval: 30000 # How long to keep dimension values in index. retentionDays: 30 # Index purge schedule (every 1h) purgeSchedule: 0 30 * * * * purgeBatchSize: 10000 # Mysql (batch mode) insertQuery: insert into DIMENSION_INDEX (NAME, VALUE, LAST_SEEN) values (?, ?, ?) on duplicate key update LAST_SEEN = values(LAST_SEEN) # Oracle # insertQuery: insert into DIMENSION_INDEX (NAME, VALUE, LAST_SEEN) values (?, ?, ?) # updateQuery: update DIMENSION_INDEX set LAST_SEEN = ? where NAME = ? and VALUE = ? # MySQL deleteQuery: delete from DIMENSION_INDEX where LAST_SEEN < ? LIMIT ? # Oracle # deleteQuery: delete from DIMENSION_INDEX where LAST_SEEN < ? and ROWNUM <= ? # Storage Buffer configuration buffers: - factType: default insertBatchSize: 1000 updateBatchSize: 200 idleTime: 2000 - factType: com.germainsoftware.apm.data.model.JavaTransaction insertBatchSize: 1000 updateBatchSize: 200 idleTime: 2000 numInsertThreads: 8 - factType: com.germainsoftware.apm.data.model.JavaMessageTransaction insertBatchSize: 1000 updateBatchSize: 200 idleTime: 2000 numInsertThreads: 8 - factType: com.germainsoftware.apm.data.model.UxSessionSegment insertBatchSize: 100 updateBatchSize: 50 idleTime: 2000 numInsertThreads: 4 - factType: com.germainsoftware.apm.data.model.SiebelWorkflowServiceTransaction insertBatchSize: 1000 updateBatchSize: 200 idleTime: 2000 numInsertThreads: 8 - factType: com.germainsoftware.apm.data.model.SiebelWebEngineTransaction insertBatchSize: 1000 updateBatchSize: 200 idleTime: 2000 numInsertThreads: 8 - factType: com.germainsoftware.apm.data.model.SiebelDatabaseTransaction insertBatchSize: 1000 updateBatchSize: 200 idleTime: 2000 numInsertThreads: 8 - factType: com.germainsoftware.apm.data.model.SiebelBusinessProcess insertBatchSize: 1000 updateBatchSize: 200 idleTime: 2000 numInsertThreads: 1 numUpdateThreads: 4 - factType: com.germainsoftware.apm.data.model.SiebelBusinessProcessStep insertBatchSize: 1000 updateBatchSize: 200 idleTime: 2000 numInsertThreads: 1 numUpdateThreads: 4 - factType: com.germainsoftware.apm.data.model.SiebelOutboundHttpRequest insertBatchSize: 1000 updateBatchSize: 200 idleTime: 2000 numInsertThreads: 4 numUpdateThreads: 4 # Management management: #address: 127.0.0.1 #security: #enabled: false endpoints: web: exposure: include: health,info,metrics base-path: / enrichment: # Every 15 minutes syncSchedule: "0 0/15 * * * *" syncSystem: countTemplate: select count(1) from SYSTEM_DIMENSION where HOSTNAME = ? insertTemplate: insert into SYSTEM_DIMENSION (HOSTNAME, NAME, TYPE, ENVIRONMENT) values (?, ?, ?, ?) updateTemplate: update SYSTEM_DIMENSION set NAME = ?, TYPE = ?, ENVIRONMENT = ? where HOSTNAME = ? processors: - name: System Dimension requiredField: system.hostname key: system.hostname # Oracle #query: select NAME, ENVIRONMENT, TYPE from SYSTEM_DIMENSION where HOSTNAME = lower(?) and ROWNUM < 2 # Mysql query: select NAME, ENVIRONMENT, TYPE from SYSTEM_DIMENSION where HOSTNAME = lower(?) limit 0,1 mapping: - system.name - system.environment - system.type - name: Target Dimension requiredField: target.hostname key: target.hostname # Oracle #query: select NAME, ENVIRONMENT, TYPE from SYSTEM_DIMENSION where HOSTNAME = lower(?) and ROWNUM < 2 # Mysql query: select NAME, ENVIRONMENT, TYPE from SYSTEM_DIMENSION where HOSTNAME = lower(?) limit 0,1 mapping: - target.name - target.environment - target.type - name: User Dimension requiredField: user.name key: user.name # Oracle #query: select DEPARTMENT, USERGROUP, USERROLE, USERTYPE from USER_DIMENSION where USERNAME = lower(?) and ROWNUM < 2 # Mysql query: select DEPARTMENT, USERGROUP, USERROLE, USERTYPE from USER_DIMENSION where USERNAME = lower(?) limit 0,1 mapping: - user.department - user.group - user.role - user.type # Disabled by default - only use if SIEBEL_DIMENSION has been populated. #- # name: Siebel Dimension # requiredField: siebel.screen # key: siebel.screen + '|' + siebel.view + '|' + siebel.applet # Oracle #query: select ALT_SCREEN, ALT_VIEW, ALT_APPLET from SIEBEL_DIMENSION where ID = ? and ROWNUM < 2 # Mysql # query: select ALT_SCREEN, ALT_VIEW, ALT_APPLET from SIEBEL_DIMENSION where ID = ? limit 0,1 # mapping: # - siebel.altScreen # - siebel.altView # - siebel.altApplet # System location lookup - name: System Location Dimension requiredField: system.hostname key: com.germainsoftware.net.IPUtils.convertToIpv4(system.hostname) # Oracle #query: select CONTINENT, COUNTRY, REGION, CITY from LOCATION_MAPPING_IPV4 where ? >= IP_FROM and ROWNUM < 2 order by IP_FROM desc # Mysql query: select CONTINENT, COUNTRY, REGION, CITY from LOCATION_MAPPING_IPV4 where ? >= IP_FROM order by IP_FROM desc limit 0,1 mapping: - system.location.continent - system.location.country - system.location.region - system.location.city # System location lookup for inbound requests - name: Source Location Dimension requiredField: source.hostname key: com.germainsoftware.net.IPUtils.convertToIpv4(source.hostname) # Oracle #query: select CONTINENT, COUNTRY, REGION, CITY from LOCATION_MAPPING_IPV4 where ? >= IP_FROM and ROWNUM < 2 order by IP_FROM desc # Mysql query: select CONTINENT, COUNTRY, REGION, CITY from LOCATION_MAPPING_IPV4 where ? >= IP_FROM order by IP_FROM desc limit 0,1 mapping: - source.location.continent - source.location.country - source.location.region - source.location.city # Disabled by default - enable to store location info on user instead of system # - # name: User Location Dimension # requiredField: system.hostname # filter: "user != null" # key: com.germainsoftware.net.IPUtils.convertToIpv4(system.hostname) # Oracle #query: select CONTINENT, COUNTRY, REGION, CITY from LOCATION_MAPPING_IPV4 where ? >= IP_FROM and ROWNUM < 2 order by IP_FROM desc # Mysql # query: select CONTINENT, COUNTRY, REGION, CITY from LOCATION_MAPPING_IPV4 where ? >= IP_FROM order by IP_FROM desc limit 0,1 # mapping: # - user.location.continent # - user.location.country # - user.location.region # - user.location.city # NodeJS extract service name # - # name: NodeJS Service Name # requiredField: path # filter: "type == \"NodeJS:Outbound\"" # key: com.germainsoftware.data.PatternUtils.extract(path, ".*/(\\w+)$", 1) # Oracle #query: select NAME from MARCOJS_SERVICE where PATH = ? and ROWNUM < 2 # Mysql # query: select NAME from MARCOJS_SERVICE where PATH = ? limit 0,1 # mapping: # - name anonymization: enabled: false fields: - user.name

Realtime Metrics (rtm-services.yml)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 # General service properties server: port: 8096 rtm: name: Germain APM Realtime Metric Services rootDirectory: var/buffers/rtm timestampExtractor: com.germainsoftware.rtm.converters.BeanExtractor typeExtractor: com.germainsoftware.rtm.converters.BeanExtractor persistenceSchedule: 30 */5 * * * * cubes: GenericMetric-System-Availability: typeKey: GenericMetric:System axes: - system.name measures: - value timeGranularity: 10 timeSpan: 900 GenericMetric-Database-Availability: typeKey: GenericMetric:Database axes: - database.name measures: - value timeGranularity: 10 timeSpan: 900 InternalEvent-GermainAPM-ComponentStatus: typeKey: InternalEvent:GermainAPM:Component Status axes: - node - engine - name - color timeGranularity: 60 timeSpan: 3600 InternalEvent-GermainAPM-EngineStatus: typeKey: InternalEvent:GermainAPM:Engine Status axes: - node - engine measures: - value timeGranularity: 60 timeSpan: 3600 FactCounter: typeKey: FactCounter:Fact:Counter axes: - name measures: - value timeGranularity: 60 timeSpan: 3600