Researching issues that several clients were having with slow Windows Roaming Profile logins and found that the common denominator was profiles being too large. Looked at Event Viewer and found nothing but Event ID 6005 – “The winlogon notification subscriber is taking long time to handle the notification event (Logon).” Looked at their Group Policy settings and found the folder that profiles were being saved in. Ran WinDirStat on the user.v6 folder and found some interesting details. It looks like downloads, Slack, Teams, and Zoom were taking up 13+GB of data that was then trying to be synced over the network. Looks like it is time to update the Group Policy to exclude some folders:If your company is looking to virtualize your servers or take them to the cloud, then contact us to setup migration evaluation.
GPO – Exclude directories in Roaming Profile
Open Group Policy Management
Edit the Roaming Profile policy
Open User Configuration > Policies > Administrative Templates > System > User Profiles
Enable – Exclude directories in roaming profiles
Add the following directories – Downloads;AppData\Roaming\Slack;AppData\Roaming\Microsoft\Teams;AppData\Roaming\Zoom
Ok your way out
Open Windows Explorer and navigate to the user.v6 folder and delete the following folders:
Wait 15 minutes for changes to propagate then reboot the effected machines and login again.
If your company is using roaming profiles to keep employees agile in the office, then contact us to setup a group policy evaluation.
Got a email from one of our co-managed IT / Tier3 / managed RMM clients that was having issues with DNS resolution. The network consists of a Synology NAS acting as Domain Controller / DNS Server and a VM on the Synology that runs the clients main application. Several of the workstations were having an issue where they could not browse to the IP address (\\192.168.0.11\sharename)of the application server at one time and could not browse to the UNC path (\\servername\sharename) of the same server on another day. First tried setting the external forwarders to Google DNS and the Forward Policy to Forward First, but the problem resurfaced. So we dug deeper into the DNS settings and found the following:
If you look closely the IP address of the server is 192.168.0.11 and the records for DNS servers associated with the domain above and below it point to servers outside the subnet of the application server (10.0.0.2). Upon further investigation this DNS server address was blocked by the firewall because it was an old IP address scheme that was no longer in use. The current good DNS server IP addresses are 192.168.40.10 and 192.168.0.10.
Turns out the stale DNS records were the problem. Made the needed changes to the DNS records and things are working great.
If your company needs a little extra help running the IT department, then contact us to setup a co-managed IT evaluation.
A bill in Congress has been brewing since October 2020 and finally passed in December 2020. Representative David Scott introduced H.R.8620 which is stated to:
“To permit payments for certain business software or cloud computing services as allowable uses of a loan made under the Paycheck Protection Program of the Small Business Administration.”
What PPP can do for you
This bill was an amendment to the Small Business Act that changes the definition of how PPP loan moneys can be used. The changes are as follows:
“the term ‘covered operations expenditure’ means a payment for any business software or cloud computing service that facilitates business operations, product or service delivery, the processing, payment, or tracking of payroll expenses, human resources, sales and billing functions, or accounting or tracking of supplies, inventory, records and expenses”
So what does this mean for your business? That you can apply for the PPP funds then use them to upgrade your out-of-date software that runs your company or use the funds to move your business into the cloud. There has never been a better time or excuse to discuss the possibilities of moving your business to the cloud and implementing those upgrades that have waited so long. By doing so you will position your company better for the Work From Home trend and be prepared for business expansion once the pandemic is over.
Here are some lessons learned from a recent recovery of a server with the following error:
Lesson #1 – Blinking Hard Drives
So when I got to the customer site the Dell server had blinking hard drive lights on two of the drives. Based on the support article about it the lights meant -“Identifying drive or preparing for removal.” and digging into the RAID controller I found the worst possible scenario for a RAID-5 array – two dead hard drives. I removed the two dead drives, cleared the configuration on the RAID controller, built a new RAID-5 array out of the remaining drives (4 out of 6), and did a fast initialize.
Lesson #2 – Drive letters on Windows Server Backup
Not sure if anyone else has noticed, but when Windows Server Backup is setup to use an external drive it likes to hide the drive by not assigning it a drive letter. This caused a few issues with the restore done from Windows Server 2012 R2 USB boot media as it couldn’t find the drive. I had to connect the external drive to my laptop then give it a drive letter. Plugged it back into the server and rebooted.
Lesson #3 – Patience is a virtue in Scanning for System Image Disks
Following the basic instructions for doing a Windows Server Backup 2012 Restore via Windows Server 2012 R2 USB boot media it came to the point where it does the scanning for System Image Disks. Turns out this can take hours depending on the speed of the drive plus the size and quantity of restores you have on the external drive. Just wait for the process to complete.
Lesson #4 – UEFI or Legacy BIOS matters
So you waited all that time for the Scanning for System Image Disks to complete and now that precious moment arrives when you realize that the Windows Server 2012 R2 USB boot media that you created was UEFI instead of legacy BIOS and the restore fails telling you so. Make sure that when you create the Windows Server 2012 R2 USB boot media that you change the settings to match the system that you are trying to restore.
Hope that these lessons help a few other Windows Server admins, who are trying to do a Windows Server Backup 2012 Restore, save some time and frustration. If you are looking for a better way to do backup and restore then contact us for details.
Farmhouse Networking continues to make strides in providing our customers with the best, most cost effective, and environmentally friendly computing experience possible. Part of that process is what has come to be known as Lifecycle Management. Each piece of hardware has an expected amount of time in which it is cost effective to use and support it. Once this time frame has been exceeded the cost of supporting the device becomes greater than the cost as shown in the following graph:
FHN Lifecycle Management
So the question remains what to do with the old computers when the time comes to replace them. Previously here in Grants Pass, OR we could support a local charity by taking them to Southern Oregon Aspire to have the computers dismantled and hard drives shredded. Now that their doors are closed we are stuck with dropping them off at the local dump, but what if you could make money while being responsible with the environment?
Farmhouse Networking is now partnering with a company called Arcoa, who do just that. Here is what they do in their R2 rated responsible recycling facility:
“We help you recover value from retired electronic equipment through responsible methods of reuse and recycling. Resale offers the best potential for value recovery, but the fast pace of innovations in technology and short product life cycles can limit equipment’s potential for reuse. From there, the best option may be to recycle the items in an environmentally friendly manner. We’ve built a robust de-manufacturing process to offer additional options for asset value recovery by disassembling equipment for commodity grade materials, which can be diverted from landfills and be used to create new materials.”
Hard drives will be electronically wiped, magnetically degaussed, or shredded based on need. The rest of the parts will be dismantled and sold with part of the profit returning to your company to help offset the cost of buying new computers. What could be better than making money on the buy?
If your company is heading towards a hardware refresh, then make the environmentally sound choice by contacting us for assistance.
There has been information released by a security research firm called Eclypsium that there is a vulnerability dubbed Boothole in Unified Extensible Firmware Interface (UEFI) Secure Boot that would allow an attacker to completely take over a workstation, laptop, or server and be nearly undetectable. All hardware vendors will have to send out updates in the near future to patch the UEFI code to secure it against this “BootHole” vulnerability. Due to the difficulty in designing and testing these types of updates it will be some time before they are released. We will keep you posted as to the release of these updates as they become available.
If your company is concerned about security, then contact us for assistance.
Had a client receive the message “There was a problem with your roaming profile…” after logging into another computer on the domain. Research and found that a simple registry fix was available, but most fixes wanted admins to export a key from another working profile then import it into the broken one. Here is the actual registry key that was used to fix the profile:
Registry Fix for roaming profile was not completely synchronized
Open Registry Editor as the broken user and NOT administrator
Navigate to HKCU > SOFTWARE > Microsoft > Windows NT > CurrentVersion > Winlogon
Right click and choose New > String Value and give it the name ExcludeProfileDirs
Enter the following – AppData\Local;AppData\LocalLow;$Recycle.Bin;OneDrive;Work Folders
Many of our customers have been experiencing some of their users having Outlook crashing immediately after opening. We even had other tech companies call to find out how we were fixing it, so we investigated and found the following known issue from Microsoft:
Users experiencing Outlook connection issues and crashes EX218604, Exchange Online, Last updated: July 15, 2020 10:12 AM Start time: July 15, 2020 9:18 AM User impact: Users may experience crashes or may be unable to access Exchange Online via Outlook. Current status: Our initial review of the available data indicates that recently deployed updates are the likely source of the problem. We’re performing an analysis of all recent service updates to isolate the underlying cause of the problem and to determine the most expedient means to restore service.
We will be keeping our monthly clients up to date on this issue.
The COVID-19 scare and ensuing rush to remote access has us thinking security. What is more basic to security than passwords. In an effort to find a way to make passwords both secure and easy to remember, I have found a website that seems to fit the bill:
A recent briefing from the FBI’s Internet Internet Crime Complaint Center (IC3) detailed current best practices and industry standards for cyber defense. Here is a summation:
Cyber Defense Best Practices
Backups – Regularly back up data and verify its integrity. Backups are critical in ransomware; if you are infected, backups may be the only way to recover your critical data.
Training – Employees should be made aware of the threat of ransomware, how it is delivered, and trained on information security principles and techniques.
Patching – All endpoints should be patched as vulnerabilities are discovered. This can be made easier through a centralized patch management system.
Antivirus – Ensure anti-virus and anti-malware solutions are set to automatically update and that regular scans are conducted. Centrally managed is even better.
File Permissions – If a user only needs to read specific files, they should not have write-access to those files, directories, or shares. Configure access controls with least privilege in mind.
Macros – Disable macro scripts from Office files transmitted via email.
Program Execution Restrictions – Implement software restriction policies or other controls to prevent the execution of programs in common ransomware locations, such as temporary folders supporting popular internet browsers, and compression/decompression programs.
Remote Desktop Protocol – Employ best practices for use of RDP, including use of VPN, auditing your network for systems using RDP, closing unused RDP ports, applying two-factor authentication wherever possible, and logging RDP login attempts.
Software Whitelisting – Implement application whitelisting. Only allow systems to execute programs known and permitted by security policy. This one takes careful planning.
Virtualization – Use virtualized environments to execute operating system environments or specific programs. No physical access to servers makes hacking harder.
Network Segmentation – Implement physical and logical separation of networks and data for different organizational units. Keep guest traffic out of your business network.
No Saved Passwords – Require users to type information or enter a password when their system communicates with a website. Better yet use a password management tool.
If your company is going to use full disk encryption or has compliance requirements that you need consulting for, then contact us for assistance.