Intrado Notifications
Monitoring - Due to the Coronavirus outbreak, World Health Organizations & Government Authorities have made recommendations for employees from certain countries within the Asia Pacific region to work from home in order to reduce the risk of the virus spreading; this is primarily in South East Asia & China.

As a result, the public will be communicating via telephone on a more frequent basis and will increase the number of calls handled by telephone carriers across the Asia Pacific region.

Increased telephone traffic may cause congestion on carrier network(s) and result in users experiencing intermittent connection problems when attempting to connect to conference calls. Users may receive busy/engaged tones, dead air, or network messages.

Intrado is closely monitoring service capacity levels in coordination with carrier partners.

The local exchange carrier (caller’s carrier) will also have increased call volumes and may experience temporary congestion problems; Intrado does not have control or ownership of local exchange carrier(s) network capacity and is solely managed by the local exchange carrier(s).

Impacted users are advised to connect to conferencing services utilizing VoIP or Video options where possible.
Feb 11, 21:45 CST
Unified Communication Services (Conferencing) Operational
Cisco CCA - Cloud Connected Audio Operational
Cisco Direct Audio / Webex PSTN Operational
Integrated Audio (Dial-out and Audio Integration) Operational
InterCall Online Operational
Leaderview Operational
Operator Assisted Conferencing Operational
Reservationless-Plus Bridging - Region: Americas Operational
Reservationless-Plus Bridging - Region: APAC Operational
Reservationless-Plus Bridging - Region: Canada Operational
Reservationless-Plus Bridging - Region: EMEA Operational
Web Collaboration Tools Operational
Video Collaboration Tools Operational
UCaaS (Unified Communications as a Service) Operational
Agent Desktop Client (ADC) Operational
Carrier Outage Operational
Cisco Contact Center Enterprise (CCCE) Operational
Cloud Contact CE Operational
Cloud Contact Pro Operational
ControlMaxx Operational
Intrado Network Operational
Jabber Operational
VoiceMaxx Operational
VoiceMaxx CE Operational
Commercial / Utility Operational
Cloud Contact Center Operational
Inbound IVR Operational
Outbound Notifications Operational
Healthcare Operational
Campaign Central Operational
Engage Operational
HouseCalls Operational
RemindMe Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Scheduled Maintenance
West Maintenance Calendar Dec 31, 18:00 - May 31, 21:00 CST
Https://teamup.com/ksbjwpgbayvv17iueu

Please use the above link to access the maintenance calendar.

PLEASE NOTE:

The calendar ONLY reflects scheduled events that may result in product downtime or impact to services during the maintenance window.

ResPlus Conference bridge maintenance is based on the bridge your traffic is assigned to. To determine your company's assigned bridge, please email GQN@west.com. We will respond with the bridge(s) specific to your account.
Posted on Oct 15, 10:01 CDT
Past Incidents
Feb 28, 2020
Completed - The scheduled maintenance has been completed.
Feb 28, 04:00 CST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Feb 27, 22:00 CST
Scheduled - Start Time: 11:00 PM US – ET / 10:00 PM CT US
Duration: 6 Hours

Region Impacted: Global

Impacted Service/Application
•HouseCalls
• LabCalls
• TSync

Maintenance Details:
HealthCare SWN HouseCalls / TLINK Platform Maintenance

Impact/Customer Experience:
Minimal Impact is expected due to the timing of the maintenance window.
• Customers may receive an unhandled exception error - if attempting to process a web request during the service move
• During the actual maintenance, services will be moved from one disk cluster to backup cluster to further mitigate any possible impact.
• As services are moved, active users could experience approximately 2 minutes of downtime. This will occur two times during the maintenance activity when traffic is moved from the primary to the backup, as well as when service is returned to the primary.
• The actual time of the moves could vary depending on the time necessary to complete, but below are the estimated windows barring any unforeseen issues.
o 1st Move Window – Approx. 1:15 AM CT – 1:45 AM CT (2:15AM ET - 2:45AM ET)
o 2nd Move Window – Approx. 2:45 AM CT – 3:15 AM CT (3:45AM ET - 4:15AM ET)

Temporary Workaround:
Customers who receive exception errors should simply wait approximately 2 minutes an try again

Back Out Plan:
N/A
Feb 25, 17:34 CST
Feb 27, 2020
Completed - The scheduled maintenance has been completed.
Feb 27, 04:00 CST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Feb 26, 22:00 CST
Scheduled - Time
11:00 PM US – ET/ 10:00 PM CT

Duration
6 Hours

Region Impacted
Global

Impacted Service/Application
•HouseCalls
• LabCalls
• TSync

Maintenance Details
HealthCare LMT HouseCalls / TLINK Platform Maintenance

Impact/Customer Experience
Minimal Impact is expected due to the timing of the maintenance window.
• Customers may receive an unhandled exception error - if attempting to process a web request during the service move
• During the actual maintenance, services will be moved from one disk cluster to backup cluster to further mitigate any possible impact.
• As services are moved, active users could experience approximately 2 minutes of downtime. This will occur two times during the maintenance activity when traffic is moved from the primary to the backup, as well as when service is returned to the primary.
• The actual time of the moves could vary depending on the time necessary to complete, but below are the estimated windows barring any unforeseen issues.
o 1st Move Window – Approx. 1:15 AM CT – 1:45 AM CT (2:15AM ET - 2:45AM ET)
o 2nd Move Window – Approx. 2:45 AM CT – 3:15 AM CT (3:45AM ET - 4:15AM ET)

Temporary Workaround
Customers who receive exception errors should simply wait approximately 2 minutes an try again

Back Out Plan
N/A
Feb 25, 17:30 CST
Feb 25, 2020

No incidents reported.

Feb 24, 2020
Resolved - This incident has been resolved.
Feb 24, 18:10 CST
Update - Teams continue to monitor the processing of the backlog.
Feb 24, 17:46 CST
Monitoring - Teams have resolved the cause of the SMS notifications failure. Engineers are monitoring the processing of backlogged SMS notifications.
Feb 24, 17:19 CST
Identified - Teams identified the cause of the SMS notifications failure. Engineers are currently working on resolving the issue.
Feb 24, 16:58 CST
Investigating - We are currently investigating reports of SMS notifications only not being sent.
Feb 24, 16:40 CST
Feb 23, 2020
Completed - The scheduled maintenance has been completed.
Feb 23, 03:00 CST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Feb 22, 23:00 CST
Update - We will be undergoing scheduled maintenance during this time.
Feb 19, 08:52 CST
Scheduled - Ticket(s): CHG0079865
Date: 2/23/2019
Start Time: 12:00 AM ET
End Time: 04:00 AM ET
Data Center(s): Louisville, KY
Component(s): MARCS

Client(s) Impacted: All VoiceMaxx; VoiceMaxxCE, & MaxxConnect Customers
Services Affected: Inbound/Outbound PSTN calls

Expected Service Impact: This change is intended to confirm and validate MARCS geo-redundancy for all customers. There is a potential for some inbound/outbound call failure.
Expected Service Downtime: There is no anticipated impact/downtime. If services become unavailable, we will immediately backout to avoid interruption to services.

Maintenance Performed By: Operations UCaaS Voice Tier II and Tier III
Maintenance Approved By: Intrado Management

Maintenance Summary:
Confirm and validate that MARCS Geo-redundancy is working for all customers. During the maintenance Intrado will be performing inbound and outbound testing to ensure that the failover test is successful. We encourage our clients to perform their own tests during this 30-40-minute window. Intrado will open a ticket for each of our clients and associate it to a Master ticket so that client engagement can provide updates via your ticket throughout the maintenance window on February 23rd. If issues are encountered during the failover window, please update the ticket with the information requested in the below template so that we can capture the data necessary to investigate your issue. Understand that Intrado will review all reports of issues in the week following maintenance and provide a response post-investigation. You can also check the status of the maintenance by clicking the following link https://west.statuspage.io.



Background:
A recent incident in Louisville occurred on 9/16 bringing attention to the lack of geo-redundancy in inbound/outbound calling for VoiceMaxx; VoiceMaxxCE; and MaxxConnect customers. Additional configuration has been done to correct the issue. This change is needed to validate and confirm geo-redundancy.


Testing:
Validate that calls are traversing MARCS infrastructure in Atlanta DC without issue.

Rollback:
Immediate restore of Louisville MARCS services.


Template if issues are encountered during the maintenance:
o Date/Time
o Phone number
o MAC address
o Phone IP address
o Call experience
 Were you able to make outbound calls?
 Were you able to make inbound calls?
 Did direct dial calls work. Extension to extension?
 Did you have 2-way audio? If no, please identify which direction audio was lost.
 Did your phones re-register?
Feb 18, 15:32 CST
Feb 21, 2020

No incidents reported.

Feb 20, 2020

No incidents reported.

Feb 19, 2020

No incidents reported.

Feb 18, 2020

No incidents reported.

Feb 17, 2020

No incidents reported.

Feb 16, 2020

No incidents reported.

Feb 15, 2020
Completed - The scheduled maintenance has been completed.
Feb 15, 01:00 CST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Feb 14, 23:00 CST
Scheduled - Failover Testing for InterCall Unified Meeting
Time:
23:00 Central (Feb 15th Midnight 00:00 AM ET)

Duration:
2 HOURS

Region Impacted:
GLOBAL

Impacted Service/Application:

InterCall Unified Meeting

Maintenance Details:

This change is to test failover of swn01-iumfips-lb01/02. During this maintenance access to VIPs will be verfied with the current and standby load balancers.

Impact/Customer Experience:

Expected Impact:

Access to the VIPs will be disrupted or intermittent during the duration of the change.



Temporary Workaround:

N/A

Back Out Plan:

N/A
Feb 14, 14:32 CST
Feb 14, 2020
Resolved - Engineers have validated testing was successful. Impact is no longer observed. This incident has been resolved.
Feb 14, 18:20 CST
Identified - Intrado teams have identified an issue with Bridge 456 where participant names are not being announced when joining a conference. Audio prompts are not being played on Bridge 456 as well. Engineering is working to resolve the issue.
Feb 14, 13:58 CST
Update - Intrado teams continue to investigate this issue.
Feb 14, 12:39 CST
Investigating - Intrado teams are currently investigating an issue with Bridge 456 where participant names are not being announced when joining a conference. Audio prompts are not being played on Bridge 456 as well.
Feb 14, 11:06 CST
Resolved - Carrier engineers have confirmed that impact has been mitigated.

Intrado engineers confirm that successful connections are now establishing correctly.

UK and Germany connectivity was restored at 13:30 GMT and Turkey at 14:04 GMT.

This outage is deemed resolved.
Feb 14, 11:29 CST
Update - Testing has been successfully completed to German and UK call back numbers.

Affected users are advised to continue utilising their appropriate dial in numbers until further notice.
Feb 14, 07:33 CST
Update - Impact has been reported for Poland in addition to the earlier reports from Germany and UK.

Intrado carriers are fully engaged and continue to investigate the call paths to identify the point of failure.

Affected users are advised to continue utilising their appropriate dial in numbers until further notice.
Feb 14, 06:47 CST
Investigating - Intrado is currently experiencing a high number 'Call Me' failures when calling back to EU countries, mostly impacting UK and Germany at this time.

An incident has been opened with our carrier partner for immediate research and prompt resolution.

Affected callers are advised to utilise their local dial-in telephone numbers where available or the standard international number.

The Webex application is not affected directly by this incident.
Feb 14, 06:04 CST