Testing Simultaneous HTTP Requests in Parallel using cURL

If you're developing a web application and are worried that a similar HTTP request could come in multiple times from different users (or clients) exactly at the same time, you can see how your application will behave by using cURL (on modern versions of Linux) to create this rare (if almost impossible) circumstance:

curl --parallel --parallel-immediate --parallel-max 3 --header "Content-Type: application/json" --request POST --data '{STRINGIFIED_JSON_PAYLOAD}' --config urls.txt

The urls.txt file will contain the URL to your API endpoint you're testing the same number of times as the –parallel-max parameter.  So in our case, it would contain:

url = "https://pathtoapiendpoint"
url = "https://pathtoapiendpoint"
url = "https://pathtoapiendpoint"

Check how your application behaves and make appropriate changes to maintain concurrency if you're worried about this happening.  There are various approaches you can use to make sure concurrency is maintained such as appending the old value of the database record into your where clause when updating to see if the data has already been changed within the timeframe of the request being processed. 

Or, you can use persistent virtual columns like this:

https://stackoverflow.com/questions/54338201/mysql-prevent-insertion-of-record-if-id-and-timestamp-unique-combo-constraint 

ASP.NET MVC – The Smart Way to Prevent Cross-Site Request Forgery (CSRF) Attempts

WebAPI (AJAX XHR) and Normal POST Operations

If your ASP.NET MVC application uses some WebAPI endpoints which are called using XHR (AJAX) requests from clientside JavaScript, you can still protect against CSRF attacks by validating the origin of such a request (when it is an AJAX request) or perform the default action of validating the anti-CSRF token (for POST form requests).

I modified the below code from https://stackoverflow.com/questions/35085507/set-validateantiforgerytoken-attribute-to-get-post-for-same-action-mvc5#answer-35085970 or ARCHIVE

using System.ComponentModel;
using System.Diagnostics;
using System.Diagnostics.CodeAnalysis;
using System.Web.Helpers;
using System.Linq;
using System.Collections.Generic;
using System.Configuration;
namespace System.Web.Mvc
{
    /// <summary>
    /// Cross-Site Request Forgery (CSRF) Prevention Filter for WebAPI and Normal MVC Controllers
    /// Normal POST operations = token is checked
    /// Normal controller GET operations = ignored
    /// WebAPI requests = check to make sure they were initiated by an AJAX request from a trusted origin
    /// </summary>    [AttributeUsage(AttributeTargets.Class | AttributeTargets.Method, AllowMultiple = false, Inherited = true)]
    public sealed class ValidateAntiForgeryTokenPOSTOrAJAXOrigin : FilterAttribute, IAuthorizationFilter
    {
        private string _salt;        

        public ValidateAntiForgeryTokenAttribute2() : this(AntiForgery.Validate)
        {
        }        

        internal ValidateAntiForgeryTokenAttribute2(Action validateAction)
        {
            Debug.Assert(validateAction != null);
            ValidateAction = validateAction;
        }
        
        [SuppressMessage("Microsoft.Naming", "CA2204:Literals should be spelled correctly", MessageId = "AdditionalDataProvider", Justification = "API name.")]
        [SuppressMessage("Microsoft.Naming", "CA2204:Literals should be spelled correctly", MessageId = "AntiForgeryConfig", Justification = "API name.")]
        [Obsolete("The 'Salt' property is deprecated. To specify custom data to be embedded within the token, use the static AntiForgeryConfig.AdditionalDataProvider property.", error: true)]
        [EditorBrowsable(EditorBrowsableState.Never)]
        
        public string Salt
        {
            get { return _salt; }
            set
            {
                if (!String.IsNullOrEmpty(value))
                {
                    throw new NotSupportedException("The 'Salt' property is deprecated. To specify custom data to be embedded within the token, use the static AntiForgeryConfig.AdditionalDataProvider property.");
                }
                _salt = value;
            }
        }

        internal Action ValidateAction { get; private set; }        
        
        public void OnAuthorization(AuthorizationContext filterContext)
        {
            string validOrigins = ConfigurationManager.AppSettings["AllowedEnvironments"]; // Example in web.config <add key="AllowedEnvironments" value="https://testurl.com:4443,https://testurl.com,https://testurl2.com" />
            bool skipCheck = false;
            
            if(ConfigurationManager.AppSettings["LocalDevMode"] == "1")
            {
                skipCheck = true;
            }
            
            // In AJAX requests, the origin header is always sent (UNLESS IT'S COMING FROM THE SAME ORIGIN), so we can validate that it comes from a trusted location to prevent CSRF attacks - but if one isn't sent, we won't do anything (assume trusted)
            // In which case, we don't need to do any token checking either 🙂
            if(!skipCheck && !string.IsNullOrEmpty(validOrigins))
            {
                List<string> validOriginURLs = validOrigins.Split(',').ToList();
                if(filterContext.HttpContext.Request.Headers["Origin"] != null && !string.IsNullOrEmpty(filterContext.HttpContext.Request.Headers["Origin"].ToString()))
                {
                    string origin = filterContext.HttpContext.Request.Headers["Origin"];
                    if(!validOriginURLs.Contains(origin))
                    {
                        filterContext.Result = new RedirectResult("~/Home/InvalidRequest");
                        skipCheck = true; // Still set to true to prevent additional validation
                    }
                    else
                    {
                        skipCheck = true;
                    }
                }
            }
        
            if(!skipCheck){
                var request = filterContext.HttpContext.Request.HttpMethod;
                if (request != "GET" && (!filterContext.HttpContext.Request.IsAjaxRequest() || (filterContext.HttpContext.Request.IsAjaxRequest() && (filterContext.HttpContext.Request.Headers["X-Request-With"] == null || filterContext.HttpContext.Request.Headers["X-Requested-With"] != "XMLHttpRequest"))))
                {
                    // Do normal form POST antiforgery token check
                    if (filterContext == null)
                    {
                        throw new ArgumentNullException("filterContext");
                    }                    try
                    {
                        ValidateAction();
                    }
                    catch(Exception e)
                    {
                        filterContext.Result = new RedirectResult("~/Home/InvalidRequest");
                    }
                }
            }
        }
    }
}

Installing the Newest Version of Python 2.7.x on Older Ubuntu Systems

If you need to upgrade to the newest version of Python 2.7.x, and you're running an older distribution (like Ubuntu 14.04), use the following commands to get and compile the latest version from source (works on Ubuntu 17.04 and older – tested on Ubuntu 14.04):

sudo apt-get install build-essential checkinstall
sudo apt-get install libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev
version=2.7.18
cd ~/Downloads/
wget https://www.python.org/ftp/python/$version/Python-$version.tgz
tar -xvf Python-$version.tgz
cd Python-$version
./configure --with-ensurepip=install
make
sudo make install

Install requests and hashlib:

sudo rm /usr/lib/python2.7/dist-packages/chardet*.egg-info
sudo rm -r /usr/lib/python2.7/dist-packages/chardet
sudo rm /usr/lib/python2.7/lib-dynload/_hashlib.x86_64-linux-gnu.so
sudo rm /usr/lib/python2.7/lib-dynload/_hashlib.i386-linux-gnu.so
sudo pip install requests
sudo easy_install hashlib

You may need to create a symlink for chardet after installing it directly from pip:

ln -sf /usr/local/lib/python2.7/site-packages/chardet /usr/lib/python2.7/dist-packages/chardet

If you get the error of "ImportError: cannot import name _remove_dead_weakref" when running a pam python based authentication script after installing the new version of python, try this fix:

sudo cp /usr/local/lib/python2.7/weakref.py /usr/local/lib/python2.7/weakref_old.py
sudo cp /usr/lib/python2.7/weakref.py /usr/local/lib/python2.7/weakref.py

Getting Let's Encrypt Certbot to Work:

Now, you'll need to delete the EFF directory from the /opt directory to avoid old configuration issues that were used for your older version of python.  Once you cleanup this directory, you'll run certbot again so it can reconfigure itself. 

sudo rm -r /opt/eff.org/
sudo certbot

Old Way

jonathonf is now a very greedy person and has made his repositories private, so this method no longer works as of 12/20/2019.

sudo add-apt-repository ppa:jonathonf/python-2.7
sudo apt-get update
sudo apt-get install python2.7

Then, you'll need to cleanup a few leftover system packages manually before installing the newest version of python-pip.  If you don't do this, you'll run into problems installing some new packages using pip.

sudo rm /usr/lib/python2.7/dist-packages/chardet*.egg-info
sudo rm -r /usr/lib/python2.7/dist-packages/chardet
sudo rm /usr/lib/python2.7/lib-dynload/_hashlib.x86_64-linux-gnu.so
sudo rm /usr/lib/python2.7/lib-dynload/_hashlib.i386-linux-gnu.so

Now, you can download and install the newest version of python-pip:

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
sudo python get-pip.py

Getting Let's Encrypt Certbot to Work:

First, you'll need to install a few packages that Certbot (the Let's Encrypt client) uses:

sudo pip install requests
sudo pip install hmac

Now, you'll need to delete the EFF directory from the /opt directory to avoid old configuration issues that were used for your older version of python.  Once you cleanup this directory, you'll run certbot again so it can reconfigure itself. 

sudo rm -r /opt/eff.org/
sudo certbot

You're done.

Full list of commands (for quickly doing all of the above):

sudo -i
add-apt-repository ppa:jonathonf/python-2.7
apt-get update
apt-get install python2.7
rm /usr/lib/python2.7/dist-packages/chardet*.egg-info
rm -r /usr/lib/python2.7/dist-packages/chardet
rm /usr/lib/python2.7/lib-dynload/_hashlib.x86_64-linux-gnu.so
rm /usr/lib/python2.7/lib-dynload/_hashlib.i386-linux-gnu.so
mkdir -p /root/Downloads
cd /root/Downloads
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python get-pip.py
pip install requests
pip install hmac
rm -r /opt/eff.org/
certbot