Ttdfjt

Customer account link broken after crawl. Run site check in woorank.com. Customer account create link will throw 404 error. Works fine when FPC is disabled. This issue has been pending for a long time. Does this mean this is not critical to Magento? Is there a workaround or patch for this issue?

https://github.com/magento/magento2/issues/20255

Short Answer:

Try updating the .htaccess file to add the HTTP HEAD block below the already existing TRACE and TRACK block. Then flush the cache and 404 should not return.

############################################
## TRACE and TRACK HTTP methods disabled to prevent XSS attacks

RewriteCond %REQUEST_METHOD ^TRAC[EK]
RewriteRule .* - [L,R=405]

############################################
## HEAD HTTP methods disabled due to https://github.com/magento/magento2/issues/20255

RewriteCond %REQUEST_METHOD HEAD
RewriteRule .* - [L,R=405]

Long Answer:

Testing this issue I found the following in my access log:

"HEAD /customer/account/create/ HTTP/1.1" 404 3708 "-" "Mozilla/5.0 (compatible; woorankreview/2.0; +https://www.woorank.com/)"

This got me thinking the HTTP HEAD request was causing the 404 to get stuck in the FPC for the requested URI. To prove this open a page on a Magento 2.3.0 or 2.3.1 site, open browser developer tools, and make a HEAD request by pasting the following into the Javascript Console:

 var xmlhttp = new XMLHttpRequest();
 xmlhttp.open("HEAD", "/customer/account/create/", true); 
 xmlhttp.onreadystatechange=function() 
 if(xmlhttp.readyState==4) 
 console.log(xmlhttp.getAllResponseHeaders());
 
 
 xmlhttp.send(null); 

Which returns:
HEAD https://site.testing.domain/customer/account/create/ 404 (Not Found)

Now the customer account creation page return 404 and the FPC cache needs to be flushed to clear.

Based on the above finds I took the quick route of blocking all HEAD request using the existing .htaccess. Obviously, NGINX would require config addition. I've read about the potential traffic drawbacks of rejecting all HEAD request so I imagine a more graceful/targeted workaround could be developed.

Credit for detailing how to make HEAD requests in a browser: https://www.laurencegellert.com/2013/03/using-your-browser-as-a-dev-tool-to-test-head-requests/

Edit: added NGINX configure details

I added the following NGINX configuration to the SSL termination server block (pre-Varnish) and confirmed it returns 405 for HEAD requests. I imagine it would work added in your version of nginx.conf.sample (post-Varnish) if needed.

## HEAD HTTP methods disabled due to https://github.com/magento/magento2/issues/20255
if ( $request_method = HEAD ) 
 return 405;