Use the END mod_rewrite flag instead of L
According to the docs in https://httpd.apache.org/docs/current/rewrite/flags.html#flag_l using [L] in an .htaccess context incurs the risk of infinite recursion, which is exactly what we were seeing in production for non-existing .php files. Use [END] instead.