Reset to device, \Device\RaidPort0, was issued.

 HP, Storage, Windows Server  4 Responses »
Sep 232016
 

Well, recently one of the servers I monitor and maintain in a remote oil town recently started throwing out a Windows event log warning:

Event ID: 129

Source: HpCISSs2

Description: Reset to device, \Device\RaidPort0, was issued.

The server is an HP ML350p Gen8 (Windows Server 2008 R2) running latest firmware and management software. It has 2 RAID Arrays (RAID1, and RAID5), and a total of 6 disks.

Researching this error, I read that most people had this occur when running the latest HP WBEM providers, as well as anti-virus software. In our case, I actually tried to downgrade to an older version, but noticed the warning still occurs. While we do have anti-virus, it’s not actively scanning (only weekly scheduled scans).

In the process of troubleshooting, I noticed that under the HP Systems Management Homepage, one of the drives in the RAID1 array, had the following stats:

Hard Read Erros:  150
Recovery Read Errors:  7
Total Seeks:  0
Seek Errors:  0

I found these numbers to be very high in my experience. None of the other drives had anything close to this (in 4 years of running, only one other disk had a read error (a single one), this disk however had tons. For some reason the drive is still reporting as operational, when I’d expect it to be marked as a predicted failure, or failed.

While all online documentation was pointing towards at locks on the array by software, from my own experience I think it was actually the array waiting for a read operation on the array, and it was this single disk that was causing a threshold to be hit in the driver, that caused a retry to recover the read operation.

Called up HPE support, I mentioned I’d like to have the drive replaced. The support engineer consulted her senior engineer and reviewed the evidence I presented (along with ADU reports, and Active Monitoring health reports), the senior engineer concurred that the drive should be replaced.

Replacing the drive resolved the issue. I’m also noticing a performance increase on the array as well.

Make sure to always check the stats on the individual components of your RAID arrays, even if everything is operating sound.

“Preparing media to store backups…” – Windows Server Backup

 Storage, Windows Server  3 Responses »
Sep 102016
 

When initiating manual backups or occasionally when automatic/scheduled backups run, a user may notice that Windows Server Backup may appear to “hang” when the status is reporting: “Preparing media to store backups…”.

In some rare cases, it may actually be in a hang state, however most of the time, it’s actually consolidating and/or checking previous backups on the destination media.

To Confirm this:

Open the Task Manager as Administrator, then click on the “Performance” tab, click on “Open Resource Monitor”. Flip over to the “Disk” tab, expand “Disk Activity”, and sort by name. You should see the read requests on the destination media, you’ll also notice that it is slowly progressing consecutively through each backup set (increments of 1, accessing multiple at a time).

This confirms that the Windows Server Backup services are functioning and it is in fact running. In one case, I had 723 previous backups, and it took around 50 minutes to count from 1 to 723, and then the backup finally proceeded.

I have also seen this occur when a previous backup failed or was cancelled. This occurs with Windows Server Backup on Windows Server 2008, Windows Server 2008 R2, and Windows Server 2012 R2.

Windows 10 – There’s already an account set up to use email address

 E-Mail, Microsoft, Windows 10  7 Responses »
Sep 082016
 

If you’re like me, you probably have your Microsoft account configured the same as your e-mail address. While many people use @live.com or @hotmail.com addresses, some of us prefer to use our actual real e-mail addresses as Microsoft account logins.

Recently, I did a fresh install of Windows 10 on my Microsoft Surface Pro. After joining the Surface to my domain, and attached my Microsoft account, I went to add my Exchange account (which is the same e-mail address I use for my Microsoft account). When trying to add, I was presented with:

There’s already an account set up to use <e-mail address>. (Account Name)

This message stopped me from configuring my Exchange account with the Windows 10 Mail, Calendar, and People apps. Researching this, I noticed numerous other people reporting this problem on multiple forums, however no one had a fix.

It appears there is a conflict with the Microsoft Account (which of course has it’s own mail, calendar, and contacts), and a separate account with the same e-mail address.

To resolve this, I restarted the machine, and logged in using a different account. I then went to “System” under control panel, “Advanced System Settings”, “Advanced” tab, then “Settings” under “User Profiles”. I then proceeded to delete the user profile and restart the system. I confirmed the user profile was fully deleted and then logged back in. Now at this point, the key is to create the Exchange (or any other mail account) before you actually attach your Microsoft account to your system login account. By configuring the e-mail account first, it will avoid this issue.

PLEASE NOTE: By deleting your user profile, you delete all of the contents of the Desktop, My Documents, Music, Pictures, settings, etc… I’d only recommend this if you have either backed up, or are performing this on a fresh install where you currently don’t have any files.

Exchange 2013 – BitlockerActiveMonitoringLogs file on System Root

 Exchange Server, Microsoft  3 Responses »
Aug 202016
 

I just wanted to create a post about this file. I’m sure some admins have seen this and wondered what it was. The “BitlockerActiveMonitoringLogs” file on the system root directory, present on Microsoft Exchange 2013 servers.

I first noticed this on a clients setup, at first assuming the worst believing the system may have been compromised. However I have seen this file on multiple Exchange installs, on multiple clients, even in my own environment, and can confirm is it present no matter what the CU release level is, thus confirming it has nothing to do with being compromised.

Date modified I’m expecting reflects last system boot-up.

Surprised to see that there are no articles online regarding this file when searching for it specifically, so I decided to create this post to let you know you’re not alone, and the file probably is a system file.

HPE MSA 2040 (and 2050/2052) – SMTP/E-Mail Notification issue (through SMTP proxy/relay servers)

 Exchange Server, HP, iSCSI, Storage  38 Responses »
Jul 302016
 

I have identified and confirmed with 2 different HPE MSA 2040 SANs an issue with SMTP notifications. I’ve identified the issue with multiple firmware versions (even the latest version as of the date of this article being written). The issue stops e-mail notifications from being sent from the MSA 2040 when the SAN is configured with some SMTP relays. This issue also occurs on HPE MSA 2050 arrays, as well as HPE MSA 2052 arrays.

The main concern is that some administrators may configure the notification service believing it is working, when in fact it is not. This could cause problems if the SAN isn’t regularly monitored and if e-mail notifications alone are being used to monitor its health.

Configuration:

-MSA 2040 (2050/2052) Dual Controller SAN configured with SMTP notifications

-SMTP destination server configured as EXIM mail proxy (in my case a Sophos UTM firewall)

Symptoms:

-Test notifications are not received (even though the MSA confirms OK on transmission)

-Real notifications are not received

-Occasionally if numerous tests are sent in a short period of time (5+ tests within 3 seconds), one of the tests may actually go through.

Events and Logs observed:

/var/log/smtp/2016/06/smtp-2016-06-20.log.gz:2016:06:20-20:44:29 SERVERNAME exim-in[16539]: 2016-06-20 20:44:29 SMTP connection from [SAN:CONTROLLER:IP:ADDY]:36977 (TCP/IP connection count = 1)

/var/log/smtp/2016/06/smtp-2016-06-20.log.gz:2016:06:20-20:44:29 SERVERNAME exim-in[18615]: 2016-06-20 20:44:29 SMTP protocol synchronization error (input sent without waiting for greeting): rejected connection from H=[SAN:CONTEROLLER:IP:ADDY]:36977 input=”NOOP\r\n”

Resolution:

To resolve this issue, I tried numerous things however the only fix I could come up with, is configuring the SAN to relay SMTP notifications through a Exchange 2013 Server. To do this, you must create a special connector to allow SMTP relaying of anonymous messages (security must be configured on this connector to stop SPAM), and further modify security permissions on that send connector to allow transmission to external e-mail addresses. After doing this, e-mail notifications (and weekly SMTP reports) from the SAN are being received reliably.

Additional Notes:

-While in my case the issue was occurring with EXIM on a Sophos UTM firewall, I believe this issue may occur with other E-mail servers or SMTP relay servers.

-Tried configuring numerous exceptions on the SMTP relay with no effect.

-Rejected e-mail messages do not appear in the mail manager, only the SMTP relay log on the Sophos UTM.

-Always test SMTP notifications on a regular basis.

 Older Entries  Newer Entries