Below is the overview of the steps involved:
1. Configure Networking
2. Load SSL Certificates
3. Add Virtual Service

Polycom’s VVX family of VoIP handsets are ideally suited for customers deploying Lync 2013 Enterprise Voice to replace their legacy PBX systems. Unlike the Lync Phone Edition (LPE)handsets which run Microsoft WinCE, the VVX has it's own software running known as UCS, the latest being version 5.0.1 and newer versions are continuously being developed with enhanced feature sets based on customer demand. UCS allows for a great deal of flexibility in its configuration compared with its LPE counterparts. In fact there are thousands of parameters that can be configured but there are a limited set of these which are relevant in a Lync voice deployment which is described in this official document from Polycom. To configure these parameters a provisioning server must be setup for the phone to download the parameters from predefined configuration files. A provisioning server is simply a FTP or HTTP server. Readers unfamiliar with setting up and using a provisioning server should refer to the same document mentioned. This article covers 5 useful configuration parameters for a Lync Enterprise Voice deployment.

Microsoft's Lync Room System (LRS) is the latest addition to the family of client editions built for Lync Server 2013. LRS is optimized to bring the immersive Lync meeting experience to participants in a meeting room. With its slick surface-like user interface, LRS makes joining meetings a "one-finger-touch" experience. During the meeting, LRS brings together both in-room and remote attendees, allowing everyone to see and collaborate on content as well as view each other in full high definition video.

Microsoft provides the LRS software to certified OEM partners which include Crestron, Polycom and Smart. This article explores the various topologies which are supported by LRS and walks through provisioning an Office365 environment for LRS. Full details of deploying an LRS for both on-premise and online environments can be downloaded here.

Supported Topologies
LRS supports on-premise Lync environments, pure Office365-S environments, as well as hybrid Lync On-Premise + Exchange On-Line environments. What's not supported today are Hybrid topologies with Lync On-Line with Exchange On-Prem. As of this writing, Microsoft is also testing LRS with Lync Hosting Pack and Office365-D; supportability statements for these can be expected in the coming months. The table below summarizes the various supported topologies today:

In Part 1 of this article we walked through the steps to deploy a Lync 2013 SBA in a simulated branch office environment. Here in Part II, we will perform some tests of the SBA to verify the correct functional behavior using Microsoft-certified IP deskphones such as the Polycom CX600 LPE and the Polycom VVX310/410 Business Media phones. We also collect some performance results of the failover and failback times in this lab. The diagram for this lab environment consists of a Lync 2013 SE FE server in the "Main Site" and an Audiocodes M800 SBA in the "Branch Site". In the main site we have a Lync 2013 PC Client and a Polycom VVX410 phone and both accounts used to login are homed on the Lync SE FE server. At the branch site we have a Polycom CX600 phone and a Polycom VVX310 phone and both account used to login are homed on the SBA. Below is the diagram of this setup:

The following tests will be performed in the sequence below:
Test #1: WAN Link Up: 
    1a. Login to all phones using PIN Auth
    1b. Login to all phones using NTLM Auth
    1c. P2P voice calls between Main site and Branch Site
    1d. Multiparty voice calls between Main site and Branch Site

Test #2: WAN Link Down
     2a. Phone registration status after WAN Link is down and failover time
     2b. Initiate P2P calls within Branch site
     2c. Phone registration status after WAN Link is restored and failback time


Test #3: WAN Link Down during active calls
     3a. Initiate P2P voice call within Branch site, then WAN Link fails
     3b. Initiate multiparty voice call between Main site and Branch site, then WAN Link fails

Just to recap, the SBA is an appliance preinstalled with Lync Registrar, Mediation Server and PSTN gateway that provides survivable telephony services to branch users in the event of a WAN outage between the branch and the main data centre hosting the Lync servers. Part 1 of this article provides a walk-thru of the steps for deploying and SBA in a Lync 2013 environment. Part 2 will focus on gathering some test results of SBA failover/failback times as well as telephony performance when used with and without Response Groups. The SBA used for this lab test is the Audiocodes Mediant800 SBA and the phones used are a Polycom CX600 Lync Phone Edition and Polycom VVX600 Lync-Compatible phone handsets. Steps to deploy an SBA are well documented in both TechNet as well as in Audiocodes' installation manual and therefore will not be repeated in detail here. The main focus of this article would be the highlights of this specific lab deployment as well as the test results of the telephony performance. The SE Lync FE server role is virtualized with Hyper-V with 4 Virtual CPU's and 4GB memory.

Upgrading Audiocodes SBA to Lync 2013
The Mediant800 used for this article was originally built for a Lync 2010 SBA and thus has to be upgraded to Lync 2013 for this deployment. The upgrade method used in this lab was the USB Upgrade and Recovery procedure. This is a straightforward process and the default settings on the RecoveryUtil.ini file on the USB stick can be used. Obtain the new Lync2013-based .wim image file from Audiocodes and copy it to the USB stick. Then plug the USB stick to the USB port at the back of the Mediant800 and power up. The system should boot from the USB and start the imaging process automatically. Once imaging is complete remove the USB and allow the system to reboot.

Configuring AD and Lync Server for SBA deployment
The SBA must first be added into AD as a computer object using the AD
Users and Computers tool. In this lab our computer object settings are as shown 
on the right. The Default Admins group is chosen as per default since we are working with the Domain Admin account. Next the computer object must be added as a member of the RTCUniversalReadOnlyAdmins group. Next, we use ADSIEdit to edit the properties of the computer object and set the attribute servicePrincipalName to "HOST/<SBA FQDN> as shown below. Finally we create a new user account belonging to the RTCUniversalSBATechnicians group for performing the Survivable Branch Appliance deployment.

In my lab setup I have Exchange 2013 CU1 server with Mailbox and CAS role installed. Exchange UM is enabled and integrated with Lync Server 2013 CU1. As Exchange 2013 does not have an Edge Transport role, the lab environment uses Exchange 2010 as the Edge Transport server. Below is the simplified diagram of the setup (firewalls and reverse proxy omitted).

Everything was working fine until Lync Phone Edition (LPE) devices and VVX Lync-compatible phones were no longer able to access the Exchange Voicemail subscriber access and auto-attendant like they used to. A quick test using the Lync client also had issues accessing the Voicemail and a snooper trace showed multiple following MS-Diagnostic errors:

"ms-diagnostics: 15007;reason="Exchange Unified Messaging server did not respond to request"
"ms-diagnostics: 1038;reason="Failed to connect to a peer"
"ms-diagnostics: 15030;reason="Failed to route to Exchange Server"

However, Exchange calendar integration continues to work on the LPE and VVX phones. This suggested that UM integration was failing because of an recent attempt to update the Exchange 2013 server to CU2. That update failed during prerequisite checks with an error message saying that there were one or more Exchange 2010 servers in the organization that has not yet been updated to SP3. However the Exchange 2010 Edge transport server was already running SP3 so it was unclear why this message appeared. To resolve this issue, I performed the following steps:

1. Remove the Edge Subscription on the Exchange 2013 server using the cmdlet 
    Remove-EdgeSubscription - identity <edgeservername>

In a recent Lync 2013 implementation, I was facing an issue with A/V not working between internal and remote Lync clients trying to establish a video call. The call gets initially connected but then drops after a few seconds with a error message on the Lync client showing "Call failed due to network issues". Both users are homed on the same FE server and therefore use the same A/V Edge server for remote and federated connectivity. A snooper trace on the call flow shows the following error in the "BYE" message:
ms-client-diagnostics: 23; reason="Call failed to establish due to a media
connectivity failure when one endpoint is internal and the other is
remote";CalleeMediaDebug="audio:ICEWarn=0x80012b

Google search results on the error message shows several websites which recommend that the solution was adding an A record in the internal DNS for the External AV Edge FQDN pointing to the external NAT'ed public IP address so that internal Lync clients can contact the AV edge. This is not mentioned in any of the official MS guides and I can confirm this recommendation to be incorrect and doing so did not resolve the issue for me. There is no requirement for this record to be created in the internal DNS in order for remote or federated A/V calls to work. To troubleshoot the issue, I looked at the "183 Session Progress" message which I've reproduced below:

a=candidate:1 1 UDP 2130706431 172.21.5.64 26698 typ host
a=candidate:1 2 UDP 2130705918 172.21.5.64 26699 typ host
a=candidate:2 1 UDP 2130705919 10.230.78.13 22972 typ host
a=candidate:2 2 UDP 2130705406 10.230.78.13 22973 typ host
a=candidate:3 1 TCP-PASS 174455807 140.242.214.33 52910 typ relay raddr 172.21.5.64 rport 29837
a=candidate:3 2 TCP-PASS 174455294 140.242.214.33 52910 typ relay raddr 172.21.5.64 rport 29837
a=candidate:4 1 UDP 184547839 140.242.214.33 57234 typ relay raddr 172.21.5.64 rport 18984
a=candidate:4 2 UDP 184547326 140.242.214.33 55494 typ relay raddr 172.21.5.64 rport 18985
a=candidate:5 1 TCP-ACT 174847999 140.242.214.33 52910 typ relay raddr 172.21.5.64 rport 29837
a=candidate:5 2 TCP-ACT 174847486 140.242.214.33 52910 typ relay raddr 172.21.5.64 rport 29837
a=candidate:6 1 TCP-ACT 1684796927 172.21.5.64 29837 typ srflx raddr 172.21.5.64 rport 29837
a=candidate:6 2 TCP-ACT 1684796414 172.21.5.64 29837 typ srflx raddr 172.21.5.64 rport 29837

From the excerpt above, we can see that the ICE negotiation had started and IP-Port candidates are being send to the other Lync client. The Lync client on the other end of the call also sends it's candidates list in a 182 Session Progress message thereafter which the two Lync clients will attempt connectivity checks to determine which of the candidates can be used. After connectivity checks have passed then the call should proceed with a 200OK message containing the selected working candidates but in the SIP trace, there is no such 200OK message. Instead the call is dropped with a BYE message containing the error message above. This means that connectivity checks have failed for some reason which could be the result of firewall ports being blocked. A check with the firewall admin revealed that port 3478 UDP was not open bi-directionaly on the AV Edge external interface. This would certainly impact AV Edge connectivity.

True enough, after opening UDP 3478 Bi-Directional on the AV Edge external interface, calls started working normally. SIP trace shows the following 200OK message after the 183 Session Progress:

a=candidate:5 1 UDP 1862270207 10.222.209.53 50105 typ prflx raddr 192.168.2.100  rport 28440
a=candidate:5 2 UDP 1862269950 10.222.209.53 59886 typ prflx  raddr 192.168.2.100 rport 28441

Instead of a just a server reflexive address (srflx) presented in an INVITE or 182 Session Progress message, we now see a peer reflexive address (prflx) in a 200OK message which points to the AV Edge pre-NAT IP address. One additional point to note is that the port range UDP 50,000-59,999 is NOT required to be open on the AV Edge external interface, unless federating with OCS2007. Both OCS2007R2 and Lync 2010/2013 do not require this port range to be open since these versions support the use of tunneling media ports through 3478 or 443. To understand more, I would recommend watching the TechEd2012 Lync Deep Dive session on ICE available at http://channel9.msdn.com/Events/TechEd/Europe/2012/EXL412

A good place to start understanding how the Mobility Service works is here: http://technet.microsoft.com/en-us/library/hh690030.aspx. Most of the steps are already provided in Technet but I find that many of the key essential must-do's are scattered in various pages rather than being consolidated in a single location.  Hence, this article serves as a companion to the Technet documentation and is intended for anyone who is encountering difficulty in successfully getting the Mobility service to work. It will also be useful in case I ever need to deploy the Mobility Service again in a different environment and do not wish to spend time to troubleshoot the commonly encountered issues. The diagram below reproduced from Technet is very helpful to understand the how the mobility service works:

During Automatic Discovery, mobile devices will first use DNS lookup to the internal DNS record lyncdiscoverinternal.<internal domain>. If not found, it means that the client is on an external network and will then lookup the external DNS record lyncdiscover.<sipdomain>. A mobile device that is internal to the network connects to the internal Autodiscover Service URL, and a mobile device that is external to the network connects to the external Autodiscover Service URL. For split-brain DNS environment, the internal autodiscover DNS record should exist in the internal DNS and not in the external public DNS. Vice versa, the external autodiscover DNS record should exist in the external public DNS and not in the internal DNS.

With the release of Exchange 2013 CU1, we can finally deploy the Unified Contact Store (UCS) into an existing Exchange2010 SP3 environment. This is because Exchange2013 is required for UCS and the GTM version of Exchange2013 did not support coexistence with previous versions of Exchange. To recap, UCS enables users to store all contact information in Exchange 2013 so that the information is available globally across Lync, Exchange, Outlook, and Outlook Web Access. Without UCS, Lync2013 clients cannot change or upload their photos. Below is the UCS architecture diagram for reference:

Deploying Exchange2013 CU1 itself is beyond the scope of this article and the steps are provided in Technet. For existing Exchange2010 environments as in the case of this lab setup, upgrading to Exchange2010 SP3 is required which also performs the prepare AD schema, thereby eliminating the need to run setup.exe /PrepareSchema using Exchange2013. After deploying Exchange2013 we also need to run the post-install steps as described in here. Most notably would be the need to assign a CA issued certificate for the CAS role as Exchange2013, like it's predecessors, by default uses a self-signed cert which Lync2013 will not recognize. Once the prerequisites are done, enabling unified contact store in Lync Server 2013 does not require any topology settings. All that's needed is that the Unified contact store policy is enabled (default is enabled), user's mailboxes have been migrated to Exchange2013, and user log in with using the Lync 2013 rich client at least once.

The Polycom CX600 is a Lync Phone Edition (LPE) device that is optimzed for Lync2010 and Lync2013 environments. As briefly mentioned in the KB Article for the January2013 CU for LPE devices, this new firmware update actually allows the phones to support Lync Online and Office365. This article descibes how to configure the CX600 for use in Office365 Plan E4 in a lab environment. For Lync On-Premise scenarios, fellow MVP Jeff Schertz has written excellent blogs on how to configure these phones for Lync available here.

Since the phones require users to be enabled for Enterprise Voice, only Office365 Plan E4 or Lync Online Plan 3 can be used with the Lync Phones as only these plans include the Enterprise Voice" feature equivalent in Lync On-Premise, which is refered to as "Lync-to-Phone" in Lync Online. In US and Canada, the "Lync-to-Phone" PSTN connectivity is provided by service provider JahJah and this service is not currently available anywhere else in the world. For Office365 users outside US and Canada, PSTN connectivity can still be achieved by deploying an On-Premise Lync environment with PSTN connection via a qualified gateway while still using Lync and Exchange Online in the cloud. These are refered to as Hybrid Lync Server deployments. Details on how to configure phones for these hybrid scenarios will be covered in a future article. In this article we will not configure any PSTN connectivity.

To allow the CX600 phones to retrieve VoiceMail and Call Logs, the Exchange Online service must be configured for Unified Messaging UM). To get started, navigate to the Exchange Admin Center and select Unified Messaging on the left tab. Then Click on the "+" button to add a new UM Dial Plan  as shown below:

On the new UM Dial Plan window, we enter a desriptive name for the dial plan followed by the Extension length which would be the number of digits in the full E.164 number excluding the "+". For eg if the country code is 65 followed by another 8 digits, then extension length would be 10 digits. Next select the "SIP URI" Dial plan type and select the desired audio language. Sample screen shot is shown below:

In a previous article, we discussed the steps to deploy a new Lync2013 Edge server into a Lync2013 FE pool that was coexisting with a legacy Lync2010 Pool with Lync2010 Edge. Users homed on the Lync2013 FE pool continued to use the legacy Edge server for remote access and federation. Although having both the legacy Edge and the 2013 Edge providing remote access is possible, such a configuration would be more difficult to manage and moreover, only one Edge can be used for federation. In this article, the steps to migrate completely to the new 2013 Edge will be discussed. Users still homed on the legacy Lync2010 FE pool as well as those homed on the new Lync2013 FE pool  will use the new 2013 Edge for Remote Access and Federation. The end result should look like the diagram below:

In a production environment, changing the federation route and media traffic route requires downtime for the Lync Server 2013 and Lync Server 2010 Edge Servers. Federated access will be unavailable for the duration of the outage.

In topology builder, edit the properties of the Lync2013 FE and associate the Edge pool with the new 2013 Edge:

In a previous article, the steps to introduce a Lync2013 Standard Edition FE server into an existing Lync2010 environment was discussed. That existing Lync2010 environment also had an Edge server for remote user access and federation. This article has 2 parts: Part I describes the tasks to connect the Lync2013 FE server to the legacy Lync2010 Edge pool. Part II then describes how to deploy a Lync2013 Edge server in parallel with the legacy edge.
A separate article will be posted later to discuss how to completely migrate all users to the Lync2013 Pool and decommission the legacy Lync2010 components.

In order to use the federated route that is being used by Lync2010, Lync2013 must be configured to use this route. This is a simple 2-step process. In the Lync2013 Topology Builder, right click the Site and select "Edit Properties..". In the Edit Properties window, click on the "Federation route" in the left pane. Then select the "Enable SIP federation" checkbox and choose the legacy Lync2010 Edge server from the drop down as show below:

Next, right click the Lync2013 FE node and select "Edit Properties". In the "Edit Properties" select the "Associate Edge pool (for media components)" check box and choose the legacy Edge from the drop down list as shown below:

Finally publish the topology and wait several minutes for the topology to be replicated across all servers. Then verify that users homed on the Lync2013 FE can federate with external users via the legacy Edge. At this stage, all remote access and federation capabilities are routed via the legacy Edge (Director shown for illustration only) as shown in the diagram below:

To deploy the Lync2013 Edge in parallel with the legacy environment, there are 5 steps to be performed:
Step 1: Define the new Edge pool in Topology Builder
Step 2: Install Prerequisite software for Lync2013 Edge
Step 3: Deploy the Edge server components
Step 4: Request and Assign Certificates for Edge
Step 5: Start Edge Services

First, create a DNS A record for the Edge internal NIC. In this lab we use edge2013.apbeta.local as the FQDN of the internal NIC and an IP of 10.222.202.94. The 3 external FQDNs for Access, Web Conferencing and AV are access13.apbeta.local, webconf13.apbeta.local and av13.apbeta.local respectively with IP in the ranges 10.250.27.[146-148]/25. We need to input these parameters when preparing the Edge server in the next step. In the Lync2013 Topology Builder, expand the "Lync Server 2013" node and right click on the "Edge Pool"
node. Select "New Edge Pool..." to start defining the  edge pool properties:

In the "Define New Edge Pool"  window, click "Next". Since we are installing a single Edge server for this lab, select "Single Computer Pool" and enter the internal FQDN of the Edge server then click "Next":

In the "Select Features" window, do not enable Federation or XMPP federation. Federation and XMPP federation are both currently routed through the legacy Lync Server 2010 Edge Server. These features will be configured in a later phase of migration. For now, just click "Next":

Next, select the appropriate IP options and enter the 3 FQDNs of the external interface as defined earlier:

Next, enter the Edge internal IP address and click "Next" followed by the external IP address and click "Next":

Specify the next hop server to be the legacy Lync2010 FE at this stage. Migration to the new Lync2013 pool will be done later:

No association of a pool with this Edge pool is selected at this time. External media traffic will continue to route through the legacy Lync2010 Edge. This setting will be configured in a later phase of migration:

Publish the topology and verify successful completion:

Here we prepare a Hyper-V VM with 2 NIC cards and install Windows Server 2008R2 SP1 Standard Edition Full install. We then configure one NIC card for the Edge internal interface and assign the IP 10.222.202.94/24. We also configure the Primary DNS Suffix of this computer to apbeta.local and do not set a default gateway, leaving the adapter DNS settings empty. Lastly, we create persistent
static routes on the internal interface to all internal networks where Lync clients or servers running Lync Server 2013 reside and edit the HOST file to contain a record for the next hop Lync2013 FE server which was configured as the Edge Server next hop address in Topology Builder.

The other NIC card is for the external interface and we configure 3 IP address 10.250.27.[146-148]/26. The internal and external subnets must not be routable to each other. We also configure the default gateway on the external interface and would normally point to an
external DNS. Since we do not have a DNS server in the DMZ in our lab so we will use host files (shown later).

On the Windows2008R2 SP1 server, do the following in sequence:

1. Run Windows Update to get all the latest hotfixes and patches. 
2. Install .NET Framework 4.5 from http://www.microsoft.com/en-us/download/details.aspx?id=30653
3. Enable Windows Powershell Integrated Scripting Envrionment (ISE) from Server Manager->Add Features
4. Install WMF3.0 from http://www.microsoft.com/en-us/download/details.aspx?id=34595 using the correct file. For Windows 2008R2 SP1 it should be Windows6.1-KB2506143-x64.msu
5. Install Windows Identity Foundation from http://go.microsoft.com/fwlink/p/?linkId=204657 using the correct file. For Windows 2008R2 SP1 it should be Windows6.1-KB974405-x64.msu

Before installing the Edge components, we need to export the topology into a file so we can import into the Edge server. In the Lync2013 Lync Management Shell, run the following cmdlet: "Export-CsConfiguration -FileName c:\config.zip" and copy this file to the Edge server.

Now insert the Lync2013 Server installation DVD into the VM and double-click the icon to autorun the setup. Click Yes when prompted to install Microsoft Visual C++ Redistributable, In the next dialog box, accept the default Installation Location and click "Install". Accept the terms in the license agreement and click "OK" to proceed. In Deployment Wizard click "Install or Update Lync Server System"

After determining the deployment state, click "Run" in "Step 1. Install Local Configuration Store":

In the Configure Local Replica of Central Management Store dialog box, choose "Import from a file (Recommended for Edge Servers)" and specify the location of the exported topology configuration config.zip file and then click "Next":

Click "Finish" when the Local Configuration Store installation is completed. Next click in "Run" in "Step 2 Setup or Remove Lync Server Components".

In the "Setup Lync Server Components" wizard, click "Next". Then and allow the wizard to complete. There's no input required in this step as the wizard will read the configuration file and install the necessary components. Click Finish to complete the wizard when finished:

First, download the Root CA Chain from the internal CA website, save the chain .p7b file locally and then use the certificates mmc to import the .p7b file into the Edge server's local computer Trusted Root Certification Authorities store:

Next, on the Lync Server Deployment Wizard, click on "Run" in "Step 3 Request, Install or Assign Certificates" and in  the Certificate Wizard, select "Edge Internal"  click "Request":

In the Certificate Request wizard, choose to "Prepare the request now, but send it later..." and in the next screen, enter a local file to save the request and then click "Next", enter a Friendly Name, then the Orgnization info, Geographical info, leave the remaining screens as default until the wizard is completed:

Next navigate to the internal CA website and click on "Request a certificate", followed by "advanced certificate request" and then "Submit a certificate request by  using a base-64-encoded CMC or  PKCS #10 file, or submit a renewal request by  using a base-64-encoded PKCS #7 file.". In the web browser paste the contents of the certificate request file created earlier into the text box and click Submit as shown below:

Next, download the certificate and save locally to a file:

Then, back on the Lync Certificate Wizard, click on "Import" to import the downloaded certificate file:

In the "Import Cerfificate" wizard, select the saved certificate file and click "Finish" to completed the wizard:

After importing the certificate, we need to click on "Assign" in the certificate wizard:

In the Certificate Assignment wizard, we click "Next" and "Finish" to complete the assignment:

After getting the internal certificate for the Edge, we now focus on the External Edge. The process is similar and not all the steps will be repeated, but only the differences will be highlighted. On the Certificate Wizard, select "External Edge certificate" and click "Request":

The remaining steps are similar to those for the internal certificate. Of course a different Certificate Request File should be specified as well as a different Friendly Certificate Name. But the Private Key must be marked as exportable as shown below:

In the Subject Names windows, note that the two SANs are already added automatically:

An additional step is required to select the SIP domain to be added to the SAN list:

In the next step, the lab does not require any additional SANs. Proceeding with the Request Summary page will allow us to generate the CSR which is saved to a file c:\extreq.req

Using the same method as the internal certificate, submit the request to the CA and save the certificate. Then import the certificate using the certificate wizard with the "Certificate file contains private key"checkbox cleared:

Click next to continue and complete the wizard, verifying that the certificate was imported successfully. Back on the certificate wizard, select "External Edge certificate" and click "Assign":

In the "Certificate Assignment" window click "Next". In the "Certificate Store" window, select the external certfiicate and click "Next":

In the "Certificate Assignment Summary" window, click "Next". Then verify successful assignment and click "Finish" to end the wizard. Both internal and external certificates should now have a green tick as shown below:

Finally, we start the services by clicking "Run" at the "Start Services":

Verify services are started successfuly and click "Finish" to end the wizard. At this stage, we have introduced a Lync2013 Edge into the mixed environment but are still using the legacy Lync2010 edge for all external access and federation services. In a future article, we will migrate the Remote Access and Media Traffic for Lync2013 users to use the Lync2013 Edge services.

Most of us who have done troubleshooting and analyzed SIP traces on Lync2010 will be familiar with the
Lync Server Logging Tool and Snooper. However, there are significant changes in the way Lync Server 2013 provides logging and analysis for troubleshooting. The Lync Server Logging Tool is now gone and replaced by the Centralized Logging Service (CLS). This article expores the CLS architecture and how to conduct basic logging and SIP trace analysis for the new Lync2013 server.CLS enables IT administrators to manage logging and search logs across all Lync servers in a deployment centrally rather than on individual servers. This is an advantage as one no longer needs to enable logging and having to look at traces on multiple front end servers in a pool. Now, we can start, stop and flush trace logging for any or all machines in the deployment from a single location. Administrators can also turn on logging based on scenarios on a per pool/machine for the entire deployment. From a centralized location, we can then search and trace these logs based on specified parameters.

Two main components form the CLS:
1. CLS Agent - runs on every Lync server and controls logging based on commands from the CLSController. It also manages log files for drive space usage  and moves old logs to a fileshare. The ClsAgent listens for commands on the following ports: TCP 50001, 50002 and 50003

2. CLSController - sends start, stop, fiush and search commands to all CLSAgents in the deployment. It aggegrates search results from CLSAgents and is available on every Lync server in C:\Program Files\Common files\Microsoft Lync Server 2013\CLSAgent

CLS logging is performed based on scenarios and there are a set of built-in scenarios that specify a group of components and log levels to be started and stopped together. Some examples of these scenarios include:

To find out specific details on a particular scenario, we can use the Lync Management Shell cmdlet Get-CsClsScenario cmdlet as shown below:

There's a special "AlwaysOn" scenario available that can be turned on all the time. It logs at the "INFO" level for many common components used in troubleshooting. By using "AlwaysOn", the idea is that admins don't need to reproduce the issue again, but rather that when an issue occurs there will be enough info in the component logs to debug. If the logs from "AlwaysOn" are not sufficient, then you still need to turn on the specific scenario relevent to the issue, reproduce the issue and get a higher level of logging. At any given time,
one additional scenario can be enabled along with "AlwaysOn".

OK enough about the architecture and let's get started using CLS to capture some logs. Let's first start AlwaysOn logging for the entire deployment followed by another scenario for a specific pool. Finally, we get the current SipStack log for a specific pool:

ClsController.exe -start -scenario AlwaysOn
CLSController.exe -start -scenario IncomingAndOutgoingCall -pools lync2013.apbeta.local
CLSController.exe -search -components Sipstack -pools lync2013.apbeta.local > sip.log

The resulting sip.log file can then be viewed using the familiar Snooper.exe tool but first, lets download and install the Microsoft Lync Server 2013 Debugging Tools from http://www.microsoft.com/en-us/download/details.aspx?id=35453. This will install snooper.exe and other tools in C:\Program Files\Microsoft Lync Server 2013\Debugging Tools. Run snooper and open the sip.log file created earlier to see the logs:

One of the coolest features about the new snooper is the ability to display the call flow diagram for a particular call. Simply select the invite message of the call and hit the Show Call Flow button. Being able to see the call in a diagram really helps to troubleshoot the problem much more easily Hovering the mouse over a message also displays additional details as shown in the diagram below:

If the log files don't contain the information you think it should contain from an issue you were trying to reproduce, you may need to flush the logs using the ClsController -flush cmdlet to make the latest logs available for searching immediately.

The location of the Lync2013 client log files have changed and are now located at %userprofile%\AppData\Local\Microsoft\Office\15.0\Lync\Tracing.

The new CLS in Lync2013 provides a superior logging capability and user experience over its predecessor in Lync2010. For more details on how to use and manage the CLS, refer to this website http://technet.microsoft.com/en-us/library/jj688101.aspx. Happy logging!

Lync2013 introduces some exciting new disaster recovery abilities where two or more Front End pools across two geographically dispersed sites can be "Paired" together for redundant failover. Each site can contain a Front End pool which is paired with a corresponding Front End pool in the other site. Both sites are active, and the new Lync2013 Server Backup Service provides real-time data replication to keep the pools synchronized. The Backup Service is a new feature in Lync Server 2013, designed to support the disaster recovery solution. It is installed on a Front End pool when you pair the pool with another Front End pool. There is no restriction on the distance between two data centers that are to include Front End pools paired with each other but high-speed links between them are recommended.

If the pool in one site fails, you can fail over the users from that pool to the pool in the other site, which then provides services to all the users in both pools. In addition to providing disaster recovery ability, two paired pools serve as the backup Registrars for each other. In Lync Server 2013, backup Registrar relationships between Front End pools are always 1:1 and reciprocal. This is a change from Lync Server 2010, in which  Front End pool backup relationships could be many to one. Even though backup relationships between two
Front End pools must be 1:1 and symmetrical, each Front End pool can still also be the backup registrar for any number of Survivable Branch Appliances, just as in Lync Server 2010.

This article explores in detail how to setup this pairing relationship and  puts the failover capabilities to the test in a lab environment.

For pool failover and pool failback, the engineering target for RTO is 15 minutes – the time required for the failover to happen, after administrators have determined that there was a disaster and initiated failover procedures. For pool  failover and pool failback, the engineering target for RPO is 15 minutes – the time measure of data that could be lost due to the disaster, due to replication latency of the Backup Service. All RTO and RPO numbers assume that the two data centers are located within the same world region with high-speed,
low-latency transport between the two sites

To setup Lync2013 Pool Pairing, we need to have two Lync2013 pools. If you haven't already installed Lync2013, check out my previous blog post here. In this article, I have already installed two Standard Edition FE pools as shown in the topology builder below:

Configuring the two FE Pools above to be backup for each other is relatively straightforward. From the Topology Builder, right-click one of the FE servers (any one will do) and select "Edit Properties..." In the properties window, select the "Resliency" tab on the left. Then select the "Associated backup Pool" checkbox and from the drop down list, select the other FE server as shown below:

Note that the default intervals for is 300s and 600s. For this test, we will set both to 30s. Go ahead and select the "Automatic failover and failback for Voice:" checkbox and enter 30 in both boxes as shown below:

Once the first FE server is configured, the other paired FE server will also be automatically configure the first FE server to be its backup registrar, thereby creating a 1:1 reciprocal relationship. We can verify this by looking at the properties of the other FE server. Proceed to publish the topology and verify that it was successful. Following this, we need to run local setup on each of the Front End Servers and then restart the servers just to be sure. Finally, start the CS Backup Service by running the powershell cmdlet 
Invoke-CsBackupServiceSync on one of the FE server. If you encounter some WCF permissions error running this cmdlet, you need to add yourself to the RTCUniversalServerAdmins” group, re-login to windows and rerun the cmdlet. A sucessful result looks like the screen capture below:

One of the key components of the Pool HA capability is the Backup Service that runs automaticaly on the front end pools that are in a paired relationship with another pool. The Backup Service ensure that all the necessary information in one pool pair is replicated to the other pool pair. In the event of failover, the active pool will contain the necessary database information required to hosts users from the failed pool.

The following new Powershell commands are for managing the Backup Service:

Get–CsBackupServiceConfiguration (Default sync interval is 2 minutes)
Set-CsBackupServiceConfiguration –SyncInterval 00:03:00
Get-CSBackupServiceStatus –PoolFQDN  (Get service stats)
Get-CSPoolBackupRelationship –PoolFQDN (Who am I paired with?)
Invoke-CSBackupServiceSync –PoolFQDN [-BackupModule  {All|PresenceFocus|DataConf|CMSMaster}] (Force Sync)

The Central Management store contains configuration data about servers and services in a Lync 2013 deployment. It provides robust, schematized storage of the data needed to define, set up, maintain, administer, describe, and operate a Lync 2013 deployment. It also validates the data to ensure configuration consistency. Each Lync deployment includes one Central Management store, which is hosted by the Back End Server of one Front End pool. If the pool hosting the Central Management store fails over, the Central Management store is failed over as well.

When you establish a pool pairing that includes the pool hosting the Central Management store, a backup Central Management store
database is set up in the backup pool, and Central Management store services are installed in both pools. At any point in time, one of the two Central Management store databases is the active master, and the other is a standby. The content is replicated by the Backup Service from the active master to the standby.

During a pool failover that involves the pools hosting the Central Management store, the administrator must fail over the Central Management store before failing over the Front End pool. After the disaster is repaired, it is not necessary to fail back the Central Management store. After repair, the Central Management store in the original backup pool can remain as the active master. 

Before initiating a Pool Failover test, let's have 2 Lync users make P2P video call under normal conditions. In this instance, both users are on the Lync2013 FE Pool which is configured with the Lync15 FE Pool as the backup pair. Below shows the screen capture of the call in progress.

It's now time to test the pool failover scenario. Since we will be performing a pool failover that involves the pool hosting the Central Management store, in this case Lync2013.apbeta.local, we must fail over the Central Management store before failing over the Front End pool. to do this, we run the Invoke-CsManagementServerFailover cmdlet on the still surviving FE Pool Lync15.apbeta.local:

Now that the CMS database has failed over to the other FE pool, we can initiate the actual Pool Failover using the Invoke-CsPoolFailover -PoolFQDN Lync2013.apbeta.local -DisasterMode cmdlet:

Once the pool failover has suceeded, the Lync clients are restored to normal functionality with presence enabled, all the while the video call is still ongoing and did not drop at any time during the test:

Assuming now that the first FE Pool Lync2013 has come back online and we wish to failback users to this pool, we can run the following cmdlet : invoke-CSPoolFailback -PoolFQDN Lync2013.apbeta.local. Users do not experience any outage during the failback process. We do not need to failback the CMS database anymore and it is fine to leave it running on the second Pool Lync15.apbeta.local:

In conclusion, the new Pool Resliency features in Lync2013 offer a much better user experience with its Pool Pairing and failover/failback capabilities. Add SQL Mirroring (to be discussed in a future article) and we truly have an enterprise-ready HA and DR solution offered with Lync2013 which will greatly benefit both users and administrators.