Tail a file on Windows

On almost every Unix system, we have tail -f to watch the end of *really really big* files.

When faced with a 36GB log file on Windows the tooling is often lacking.

I borrowed / adapted a little PowerShell function to extract the last n log lines from a file, and write to a new file:
https://gist.github.com/crossan007/b5e8ac4579ba61eb1967315657406751

Partially borrowed from: https://stackoverflow.com/questions/36507343/get-last-n-lines-or-bytes-of-a-huge-file-in-windows-like-unixs-tail-avoid-ti

TIL: Java code in Jenkins pipelines run on the Master

I was trying to read a file with Java.io.File in a Jenkins Groovy Scripted Pipeline on a non-master node. I kept getting an exception that the file was not found (java.io.FileNotFoundException)

Turns out that Java code written in scripted pipelines (Groovy) runs on the master node: https://issues.jenkins-ci.org/browse/JENKINS-37577. This is as-designed behavior, and accessing files in the workspace on a non-master node should use the readFile function in the Pipeline Basic Steps DSL https://jenkins.io/doc/pipeline/steps/workflow-basic-steps/#pwd-determine-current-directory

I’m thoroughly embarrassed at how many failed Jenkins jobs and alerts I’ve triggered while discovering this.

Windows 10 Password Recovery

DISCLAIMER: DO NOT EXECUTE THIS PROCESS WITHOUT EXPLICIT APPROVAL FROM THE SYSTEM OWNERS.  I AM NOT ENDORSING OR APPROVING ANY ILLEGAL ACTIVITY WHICH COULD BE ACCOMPLISHED FOLLOWING THESE STEPS

An older friend forgot his computer password; asked me for help.

I booted the machine, and saw an email address where the Windows 10 username normally would be;  my first thought was “oh, great; this is a Microsoft Online  joined computer, password recovery probably won’t happen”

I did a little research, and found some evidence that suggests my seemingly outdated knowledge about passwords being stored in the SAM seems to still stand.  However, Windows 10 Anniversary Update changed the encryption algorithm used on the SAM: https://twitter.com/gentilkiwi/status/762465220132384770

This algorithm change broke my normal tool (OPHCRACK), since it was unable to read the NTLM hashes from the SAM.  SAM encryption caused OPHCRACK to incorrectly read every account hash as 31d6cfe0d16ae931b73c59d7e0c089c0.  So, I copied the SAM and SYSTEM files (at C:\Windows\System32\config) from the target machine to my desktop for additional processing.

Mimikatz has a module `lsadump::sam` which accepts parameters for offline SYSTEM and SAM decryption.  Easy command line:

lsadump::sam /system:c:\users\charles\documents\system /sam:c:\users\charles\documents\sam

This returned decrypted NTLM hashes for easy cracking.

I decided to try a new tool here to crack the plain text password from the NTLM hashes: Hashcat.  There’s a Windows 64bit compiled version (I know, I know don’t run random binaries…) which made it easy to get cracking quickly.

I copied the hash from the output of Mimikaz into a text file called hashes.txt and ran the command

.\hashcat64.exe -m 1000 -a 3 -O -o pass1.txt .\hashes.hash

My 10 year old computer cracked the Microsoft Online account NTLM Windows 10 password hash in ~8 minutes. It was two dictionary words and a two-digit number for a total of 8 characters.  I was using brute-force in this scenario, so the fact that dictionary words were used is of no consequence.  Had I been using a dictionary, the attack would have likely concluded sooner.

Just for fun, I generated a new NTLM hash, but replacing vowels with numbers (i with 1 and the e with 3 and so fourth), the attack took the same amount of time.


import hashlib
print hashlib.new('MD4', 'password'.encode('utf-16le')).hexdigest()

Moral of the story:  USE STRONG PASSWORDS AND A PASSWORD MANAGER

Un-Approve SharePoint List Item Previous Versions

I recently had a change request against a SharePoint Forms Library I had created a few years ago – the request was to adjust the permissions so that form submitters could see only the forms that they’ve submitted (and not others).

This is a generally straightforward action on new libraries: enable ” Require content approval for submitted items?”, and change “Who should see draft items in this document library?” to

However, enabling these settings seems to have caused the items that already existed to have an Approval Status of “Approved, ” despite a pending Approval workflow.  This caused the undesired effect of allowing users who do not hold the “Approve” permission level to access previous version of items still in the approval workflow.

I needed to reject previous versions of forms where the current version had not yet been approved.  On lots of items.

I found numerous examples from google how to use PowerShell to set the Approval Status of list items; however, nearly every example dealt with only the current version of a list item – making no mention of altering the approval status of previous versions of list items.

Additionally, I found a few posts attempting to manipulate attributes for previous versions;  the responses for each of these inquires were varied:

  • “you can’t – history is read-only,”
  • “you can migrate the documents to a new list, and re-build the history”
  • “you can delete the old versions”

I even found a mega-thread on TechNet how to “List and Delete List Item Versions using PowerShell,” and a “Complete Guide to Getting and Setting Fields Using PowerShell”

None of these options accomplished what I was seeking:  to simply remove the approval on previous versions.

Finally, I resorted to simply poking at the objects from PowerShell (never under-estimate the power of Get-Member to explore objects) Attempting to modify the properties on a previous version would yeild the error message “Unable to index into an object of type Microsoft.SharePoint.SPListItemVersion” (Link)

Ok – different approach:  I know my desired action is feasible via the UI for single list items:

So, I opened Chrome developer tools and captured the command sent when clicking “Reject this version”: A POST to “/_layouts/versions.aspx” with the ItemID, and an “op” value of “TakeOffline”. A quick google search revealed a server-side object model equilavent: Microsoft.SharePoint.SPFile.TakeOffline

My solution: invoke SPListItem.File.TakeOffline() for every file which is currently pending and has a previously approved version:

 

SharePoint 2013 List Workflows Failing

Quick Post.

Today I had an issue with a SharePoint 2013 List Workflow not running on a SharePoint Online Team Site.

 

Retrying last request. Next attempt scheduled in less than one minute. Details of last request: HTTP  to https://<SomeCoolTenant>.sharepoint.com/sites/<SomeCoolSite>/_api/web/lists(guid'**********************************') Correlation Id:  Instance Id: *************************************

System.Net.WebException: The request was aborted: The request was canceled. ---> System.InvalidOperationException: Failed to fetch an access token from the token service. The token service returned an error type of 'unauthorized_client' with the following description: AADSTS70001: Application with identifier '**************************' was not found in the directory **************************************
Trace ID: **********************************
Correlation ID: *****************************************
Timestamp: 2017-10-30 14:07:03Z ---> System.Net.WebException: The remote server returned an error: (400) Bad Request.
at System.Net.HttpWebRequest.GetResponse()
at Microsoft.Activities.Hosting.Security.OAuthS2SSecurityTokenServiceCredential.FetchAccessToken(Uri stsUri, String targetServiceAudience, String authenticatorToken, HttpWebRequest request, TimeSpan timeout, EventTraceActivity eventTraceActivity, TimeSpan& expirationDuration)
--- End of inner exception stack trace ---
at Microsoft.Activities.Hosting.Security.OAuthS2SSecurityTokenServiceCredential.FetchAccessToken(Uri stsUri, String targetServiceAudience, String authenticatorToken, HttpWebRequest request, TimeSpan timeout, EventTraceActivity eventTraceActivity, TimeSpan& expirationDuration)
at Microsoft.Activities.Hosting.Security.OAuthS2SSecurityTokenServiceCredential.GetAccessTokenFromTokenService(OAuthS2SPrincipal client, OAuthS2SPrincipal targetServiceAudience, HttpWebRequest originalRequest, EventTraceActivity eventTraceActivity, TimeSpan& expirationDuration)
at Microsoft.Activities.Hosting.Security.OAuthS2SSecurityTokenServiceCredential.GetAuthorization(OAuthS2SAuthenticationChallenge[] bearerChallenges, HttpWebRequest request, EventTraceActivity eventTraceActivity)
at Microsoft.Activities.Hosting.Security.OAuthS2SAuthenticationModule.AuthenticateInternal(String challenge, WebRequest request, OAuthS2SCredential credential, EventTraceActivity eventTraceActivity)
at Microsoft.Activities.Hosting.Security.OAuthS2SAuthenticationModule.Authenticate(String challenge, WebRequest request, ICredentials credentials)
at System.Net.AuthenticationManagerDefault.Authenticate(String challenge, WebRequest request, ICredentials credentials)
at System.Net.AuthenticationState.AttemptAuthenticate(HttpWebRequest httpWebRequest, ICredentials authInfo)
at System.Net.HttpWebRequest.CheckResubmitForAuth()
at System.Net.HttpWebRequest.CheckResubmit(Exception& e, Boolean& disableUpload)
at System.Net.HttpWebRequest.DoSubmitRequestProcessing(Exception& exception)
at System.Net.HttpWebRequest.ProcessResponse()
at System.Net.HttpWebRequest.SetResponse(CoreResponseData coreResponseData)
--- End of inner exception stack trace ---
at Microsoft.Workflow.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)
at Microsoft.Activities.Hosting.HostedHttpExtension.HttpRequestWorkItem.HttpRequestWorkItemAsyncResult.End(IAsyncResult result, Int32& responseCode)
at Microsoft.Activities.Hosting.HostedHttpExtension.HttpRequestWorkItem.OnEndComplete(ScheduledWorkItemContext context, IAsyncResult result)

 

Turns out that I had forgotten to enable the Workflows can use app permissions site feature:

So – If you’re not yet using Microsoft Flow and still need those SharePoint 2013 Workflows, remember to enable this site feature.

The Un-Deletable File

I’m not sure how I created it, but somehow I managed to create a folder called

'.

I can’t delete it from Windows Explorer, PowerShell, CMD,  [System.IO]::Delete(), or any other method I’ve attempted yet.  I also can’t delete the parent folder.

SDelete won’t work: https://technet.microsoft.com/en-us/sysinternals/sdelete.aspx

 

I’ve checked for and found no open handles on the file

I cannot move the folder (or it’s parent) to another location – It’s hard-fastened to my desktop.

My drive is BitLocker encrypted, so I can’t mount it on a Linux Device for deletion, and leveraging WinPE for BitLocker decryption would  likely subject me to the same file APIs .

 

InfoPath Notify on Specific Changed Fields

Scenario:

  • Notify specific users upon the change of any field in an infopath form excluding a limited number of fields.
  • InfoPath Full Trust Form
  • SharePoint designer Workflows

I’ve seen lots of people ask for this, with a lot of  the same general response:

it’s not really possible to maintain field-specific change history, or field-specific alerting

I think that Solution 2 below could address this.

Solution 1:

set up rules on every field in the form to set “NotifyParticipatns” to 1 on change.

Pros:

  • No code required

Cons:

  • Lots of rules and room for error

Solution 2:

Global event listener for all xml fields.  Note the XPath selector used for registering the Changed event handler:

"/my:myFields/*"

Conditional for which fields should be included / excluded from the alert.

Set the “NotifyParticipants” based on conditional result.


 public void InternalStartup()
 {
 EventManager.FormEvents.Loading += new LoadingEventHandler(FormEvents_Loading);
 EventManager.XmlEvents["/my:myFields/*"].Changed += new XmlChangedEventHandler(Form_Changed);
 }

<br data-mce-bogus="1">

public void Form_Changed(object sender, XmlEventArgs e)
 {
 // Write your code here to change the main data source.
 List&lt;string&gt; AlertableFields = new List&lt;string&gt;();
 string ModifiedField = e.Site.Name.ToString();
 AlertableFields.Add("my:Street");
 if( AlertableFields.Contains(ModifiedField))
 {
 Debug.WriteLine("Alerting on " + ModifiedField );
 }
 else{
 Debug.WriteLine("Not alerting on " + ModifiedField );
 }
 
 }

 

Pros:

  • Much cleaner solution

Cons:

  • Code.
  • Full Trust

 

ADFS Username Behavior

Problem

ADFS 4.0 on Windows Server 2016 tells users to log in with their full email address “someone@example.com.”  This generates many support requests, and complaints about too much typing.

Additionally, some extranet users may have email addresses not on the domain, and it’s unclear which email address they should supply.

This affects both the ADFS log in page, and the ADFS password change page.

Solution Methodology

ADFS Server 4.0 has PowerShell cmdlets to manage the content delivered to users during authentication requests: https://technet.microsoft.com/windows-server-docs/identity/ad-fs/operations/ad-fs-user-sign-in-customization

We’ll focus on the following

Get-AdfsWebTheme

and

Set-AdfsWebTheme

Of particular interest here is that we’re able to modify the JavaScript that runs on these pages.

Steps

Use PowerShell to manage custom ADFS Themes

  1. Export the Default ADFS Theme using this snippet:
     Export-ADFSWebTheme -Name "Default" -DirectoryPath c:\test
  2. Use your  favorite editor to open c:\test\script\onload.js
  3. Add the snippets from below (as desired) into onload.js
  4. Create a New ADFS Theme
     New-AdfsWebTheme -Name BetterDefault -SourceName c:\test 
    1. Set your new theme as the default (best for testing)
       Set-ADFSWebConfig -ActiveThemeName BetterDefault 
  5. Alternatively, you may update an existing theme with your code changes
    Set-AdfsWebTheme -TargetName "Default" -AdditionalFileResource @{Uri=“/adfs/portal/script/onload.js”;Path=“C:\theme\script\onload.js"}

Placeholder Text Solution

To update the “someone@example.com” placeholder on both the login and the password change ADFS pages, paste this code into your onload.js, and update your ADFS theme.

function UpdatePlaceholders() {
    var userName;
    if (typeof Login != 'undefined'){
        userName = document.getElementById(Login.userNameInput) 
    }
    if (typeof UpdatePassword != 'undefined'){
        userName = document.getElementById(UpdatePassword.userNameInput);
    }
    if (typeof userName != 'undefined'){
        userName.setAttribute("placeholder","Username");
    }
}

document.addEventListener("DOMContentLoaded", function(){
  // Handler when the DOM is fully loaded
  UpdatePlaceholders()
});

 

Formatting of the Username field

For single-domain organizations, it may be less than desirable to force users to enter the domain name as part of their username. To “fix” this requirement of entering usernames in a format of “domain\username” or “username@domain.com”, paste the following code into your onload.js.  Make sure to update your domain where appropriate.

Logon Username Format Solution

 


if (typeof Login != 'undefined'){
    Login.submitLoginRequest = function () { 
    var u = new InputUtil();
    var e = new LoginErrors();
    var userName = document.getElementById(Login.userNameInput);
    var password = document.getElementById(Login.passwordInput);

    if (userName.value && !userName.value.match('[@\\\\]')) 
    {
        var userNameValue = 'example.org\\' + userName.value;
        document.forms['loginForm'].UserName.value = userNameValue;
    }

    if (!userName.value) {
       u.setError(userName, e.userNameFormatError);
       return false;
    }


    if (!password.value) 
    {
        u.setError(password, e.passwordEmpty);
        return false;
    }
    document.forms['loginForm'].submit();
    return false;
};
}

Password Change Username Formatting Solution


if (typeof UpdatePassword != 'undefined'){
    UpdatePassword.submitPasswordChange = function () { 
    var u = new InputUtil();
    var e = new UpdErrors();

    var userName = document.getElementById(UpdatePassword.userNameInput);
    var oldPassword = document.getElementById(UpdatePassword.oldPasswordInput);
    var newPassword = document.getElementById(UpdatePassword.newPasswordInput);
    var confirmNewPassword = document.getElementById(UpdatePassword.confirmNewPasswordInput);

    if (userName.value && !userName.value.match('[@\\\\]')) 
    {
        var userNameValue = 'example.org\\' + userName.value;
        document.forms['updatePasswordForm'].UserName.value = userNameValue;
    }

    if (!userName.value) {
       u.setError(userName, e.userNameFormatError);
       return false;
    }

    if (!oldPassword.value) {
        u.setError(oldPassword, e.oldPasswordEmpty);
        return false;
    }

    if (oldPassword.value.length > maxPasswordLength) {
        u.setError(oldPassword, e.oldPasswordTooLong);
        return false;
    }

    if (!newPassword.value) {
        u.setError(newPassword, e.newPasswordEmpty);
        return false;
    }

    if (!confirmNewPassword.value) {
        u.setError(confirmNewPassword, e.confirmNewPasswordEmpty);
        return false;
    }

    if (newPassword.value.length > maxPasswordLength) {
        u.setError(newPassword, e.newPasswordTooLong);
        return false;
    }

    if (newPassword.value !== confirmNewPassword.value) {
        u.setError(confirmNewPassword, e.mismatchError);
        return false;
    }

    return true;
};
}

Thanks for reading!  If you have any questions, feel free to send me a tweet @crossan007.

SharePoint 2016 SMTP Authentication

Edit: It appears that this has been fixed in KB 3191880 :

SharePoint outbound email messages incorrectly try to authenticate to SMTP servers that support Generic Security Service Application Program Interface (GSSAPI), Kerberos, or NTLM authentication. This may prevent email messages from being sent. After you install this update, SharePoint sends email messages anonymously without authentication.


Recently I encountered an issue where SharePoint designer workflow’s emails not being delivered.

Additional inspection revealed that the messages in question were addressed to an Exchange Distribution group with “Permitted Senders.”  This designation meant that messages sent to this distribution group must be received from an authenticated sender (which SharePoint does not support by default: SHAREPOINT 2016 OUTBOUND SMTP FAILURES).

Old Solution

One solution I’ve used in the past is to setup Microsoft’s SMTP server on one of the SharePoint servers, and use that to relay (authenticated) messages to the Exchange server.   This has generally worked fine in the past, but  has always felt a little kludgey.

Seriously, Microsoft?  You’re recommending that we install IIS6 tools on a modern server?

The Problem

Anyway, the above solution breaks down with SharePoint 2016 in certain scenarios:  When sharing documents in SP2016, the “invitation” is sent as the user who initiated the invitation!!!

By default, Exchange only allows authenticated users to send as the account who’s credentials were supplied.

This presents a “Catch 22:”

  • Enable IIS6.0 SMTP relay to send Authenticated messages to Exchange and be able to relay to groups (and external domains)
    —————————-OR—————————————–
  • Configure SharePoint to send through an unauthenticated receive connector, and be allowed to send as any user, but not able to relay otuside the domain, or to groups which require authentication.

I went down a few different solution paths trying to solve this:

Failed Attempt 1: Grant Send-As Permission to SharePoint

Attempt to grant the  SharePoint SMTP service account (since I was already sending authenticated mail) “send-as” permissions on all mailboxes in the domain.

This just felt kludgey, and I was ultimately not able to get it to work.

I may have not waited the recommended 2 hours for the Mailbox Cache Idle Limit to expire:  https://technet.microsoft.com/en-us/library/aa996988(EXCHG.80).aspx

 

Successful Attempt: Configure Externally Secured Exchange Connector

The solution for me was to create a new “Externally Secured” Exchange Receive connector: https://technet.microsoft.com/en-us/library/mt668454(v=exchg.160).aspx

Essentially, this allows the hosts defined in the receive connector’s scope to deliver “unauthenticated” SMTP traffic as if it were authenticated.  

This fulfills my SharePoint requirements:

  •  To “send-as” on behalf of users in a document sharing scenario.
  • To send email as SharePoint to distribution groups which require the sender to be authenticated
  • To send email to users outside of my domain.

I hope this helps someone (even if it’s me in the future).