Can't get application.css or application.js to cache in browser

[mcblum]

I'm pulling my hair out over this one. All JS and CSS files are cached in the browser except for the ones created by the asset pipeline. Here's my .htaccess file

<FilesMatch "\.(ttf|otf|eot|woff)$">
  <IfModule mod_headers.c>
    Header set Access-Control-Allow-Origin "*"
  </IfModule>
</FilesMatch>

<IfModule mod_rewrite.c>
    RewriteEngine On
    RewriteRule ^(.*)$ public/$1 [L]
</IfModule>

#1 YEAR
<FilesMatch "\.(flv|ico|pdf|avi|mov|ppt|doc|mp3|wmv|wav)$">
Header set Cache-Control "max-age=604800, public"
</FilesMatch>

#1 WEEK
<FilesMatch "\.(jpg|jpeg|png|gif|swf)$">
Header set Cache-Control "max-age=604800, public"
</FilesMatch>

#3 HOUR
<FilesMatch "\.(txt|xml|js|css)$">
Header set Cache-Control "max-age=10800"
</FilesMatch>

# NEVER CACHE - notice the extra directives
<FilesMatch "\.(html|htm|php|cgi|pl)$">
Header set Cache-Control "max-age=0, private, no-store, no-cache, must-revalidate"
</FilesMatch>

Environments are set correctly and the app returns 'production' when queried.

If anyone has any ideas, please let me know. We have three different apps using this code and none of them are caching.

Thank you,
Matt

[mcblum]

A look at the network tab of the Chrome dev console:

[EpicVoyage]

application.js and application.css are served through asset-pipeline, so the .htaccess file is not going to control your cache settings. Take a look at your asset-pipeline config.php file and verify that 'production' is in the 'cache' array.

Link: https://github.com/CodeSleeve/asset-pipeline#cache

[mcblum]

Thanks for your reply, @EpicVoyage. It is - here's the line copied and pasted:

    'cache' =>  array('production'),

Also, if I run php artisan env on the server I get 'production'.

[EpicVoyage]

Could you clear the cache in app/storage/cache/asset-pipeline/?

$ php artisan assets:clean

The cache should be re-built on the next page load, and after that you should receive 304 headers for these resources. If you don't, we may have to wait for the developer to reply.

[mcblum]

@EpicVoyage I cleared the cache and same result. I dug into the cache and I see now where it sets the headers but for some reason mine are not getting get. We actually have 3 apps all using Asset Pipeline and none of them are working. Is this still being actively developed?

[EpicVoyage]

Heh, that's a good point. His last reply to any of these issues seems to have been a little over 3 months ago. I'll try to look through the code after work today. I got my setup running yesterday, but it was not as easy as I would have liked.

[mcblum]

That's really nice of you, @EpicVoyage, thank you. I looked through it a couple times but I can't seem to figure out where the breakdown is.

Thanks again and let me know if you come up with anything.

[EpicVoyage]

Could you look at your config file again? There should be a setting called cache_server and a path is passed to that. Mine looks like this:

'cache_server' => new Assetic\Cache\FilesystemCache(App::make('path.storage') . '/cache/asset-pipeline'),

App::make('path.storage') . '/cache/asset-pipeline' translates into app/storage/cache/asset-pipeline on my install. Are there any files that have been stored in this folder?

If there are no files here, or if they are old, then PHP is probably having difficulty in writing files to the directory. If there are files there, open each one until you find a file that contains the application.js manifest file (this will contain your = require directives). If the application.js is combined with your other JS files then we are looking at a more complex issue. If the application.js contents are alone in a file then Asset Pipeline thinks we are in development mode.

If application.js is alone, you may wish to debug via the cache function in vendor/codesleeve/sprockets/src/Codesleeve/Sprockets/Parsers/ConfigParser.php. PHP's var_dump and die functions may be useful here.

[mcblum]

@EpicVoyage Ok so I looked through that entire directory and I see all of my JS and CSS files there, minified, but absolutely no sign of the application.js file. It's not combined with anything, it's just not in the directory.

[mcblum]

@EpicVoyage Do you have a cached, minified application.js file? I've cleared the cache, watched it disappear and still can't seem to get it to work.

[EpicVoyage]

@mcblum My apologies for disappearing on you. I had some trouble with my dev machine and then a busy weekend. Give me another day or two and I'll see what else I can come up with.

To answer your question, though, my application.js file is minified and contains all of the site's JS. The site that I am working on is not ready to be published, but I brought it up so that you can take an external look at it if you wish: http://kjv.onlyism.com/

[mcblum]

@EpicVoyage No worries at all. Hope you got everything fixed. I think your and my application.js files are the same, but I don't see that exact file in my cache directory, I see all of the individual less, js and css files there. They are all minified.

I do think this is a question that I'd love for a developer to weigh in on. It's crazy to me that I'm the only one with this problem and yet it exists in every Laravel app I've built. I must have done something...

[zub0r]

I had the same problem, but only on remote host. After some research I found out, that php build-in server uses timestamp date format for Last-Modified header line - thats why caching works on localhost if you put 'cache' => array('local'), in your config.

However, for all other web servers you need to use RFC 7231 date format:
Last-Modified: Tue, 15 Nov 1994 12:45:26 GMT

Extending ClientCacheFilter and swapping it as cache_client in config for remote servers solved the issue for me.
Same issues - #174, #171, #161

Hope this helps save somebody's neurons ;)

class CustomClientCacheFilter extends \Codesleeve\AssetPipeline\Filters\ClientCacheFilter implements ClientCacheInterface
{

    /**
     * Modified date format to work with remote server
     * @param  string $key
     * @return string
     */
    private function getLastTimeModified($key)
    {        
        $lastModified = filemtime($this->asset->getSourceRoot() . '/' . $this->asset->getSourcePath());
        return @gmdate('D, d M Y H:i:s ', $lastModified) . 'GMT';
    }

}

[mcblum]

@matohavo I haven't had a chance to try this but if you're right I'll owe you big time. Could you help me understand something - where does this code go and what modifications did you make to implement it? I've tried first changing the caching to 'local' and that basically breaks everything. I hope I can find a fix because there's a lot of unnecessary waiting when you have to serve the css and js every time :(

Thank you!

ETA: Scratch that, it works great now and is caching locally, like you said. Any pointers to get the above code to work on the server would make my day!

ETA: Oh. Man. It works. I modified the file in the vendor directory (which is dumb and will be overwritten with any updates). If you have a second and could help me understand how to use your function to extend the other one I would be grateful. This makes my day!!!

ETA: Ok so it worked for a bit, and then when I logged out and logged back in I saw that my application.js and application.css files on both apps were now empty after adding the code you wrote... I've reverted for the time being - strange though.

[mcblum]

@matohavo Any chance you have a second this week to assist with this? I feel like I'm so close :)

[zub0r]

Sure - just put this code into your project e.g. into app/extensions/assetPipeline/CustomClientCacheFilter.php

<?php namespace App\Extensions\AssetPipeline;

use DateTime;
use Assetic\Asset\AssetInterface;
use Assetic\Cache\CacheInterface;
use Codesleeve\Sprockets\Interfaces\ClientCacheInterface;

class CustomClientCacheFilter extends \Codesleeve\AssetPipeline\Filters\ClientCacheFilter implements ClientCacheInterface
{

    /**
     * Modified date format to work with hostmonster
     * @param  string $key
     * @return string
     */
    private function getLastTimeModified($key)
    {        
        $lastModified = filemtime($this->asset->getSourceRoot() . '/' . $this->asset->getSourcePath());
        return @gmdate('D, d M Y H:i:s ', $lastModified) . 'GMT';
    }

}

Then you need to tell your composer how to find it, so add that path to autoload part in composer.json:

"autoload": {
        "classmap": [
            "app/commands",
            "app/controllers",
            "app/models",
            "app/database/migrations",
            "app/database/seeds",
            "app/extensions/assetPipeline"         <<< add this
        ],

and in console run composer dump-autoload.

Now you need to swap default asset-pipeline cache filter for your custom filter. Publish asset-pipeline config with php artisan config:publish codesleeve/asset-pipeline if you haven't already.

Now if you want to apply new filter only on production server (prod environment in my case), create directory prod in app/config/packages/codesleeve/asset-pipeline/
and add app/config/packages/codesleeve/asset-pipeline/prod/config.php:

<?php
return array(   
    /*
    |--------------------------------------------------------------------------
    | cache_client
    |--------------------------------------------------------------------------
    | Client with customized getLastTimeModified format
    */
    'cache_client' => new App\Extensions\AssetPipeline\CustomClientCacheFilter,
);

If you want to apply the filter on all environments, just edit that line directly in app/config/packages/codesleeve/asset-pipeline/config.php.

Hope this helps ;)

[mcblum]

@matohavo Thank you!!! It works perfectly in terms of setting up caching, but for some reason now that the files are returning 304 headers they are blank. If I disable caching, the CSS and JS are once again combined correctly.

[evantishuk]

@matohavo How does that work with ClientCacheFilter's get function? Specifically here:

https://github.com/CodeSleeve/asset-pipeline/blob/master/src/Codesleeve/AssetPipeline/Filters/ClientCacheFilter.php#L94

if ($modifiedSince >= $lastModified)

Wouldn't your code cause that to have some unpredictable results?

[zub0r]

My custom filter extends ClientCacheFilter class, which means that it uses
get method of parent class. Only thing that changes is last modified date
format.

In what way would it be unpredictable if you know header date format of
your web server?

@matohavo https://github.com/matohavo How does that work with
ClientCacheFilter's get function? Specifically here:

https://github.com/CodeSleeve/asset-pipeline/blob/master/src/Codesleeve/AssetPipeline/Filters/ClientCacheFilter.php#L94

if ($modifiedSince >= $lastModified)

Wouldn't your code cause that to have some unpredictable results?

—
Reply to this email directly or view it on GitHub
#205 (comment)
.

[mcblum]

@matohavo @evantishuk As far as I can tell, his code works great. I still can't get it to work, though, as both of my files are blank. The blank files are, however, being served with the correct headers :)

[evantishuk]

I would expect the lastModified date, which is now a gmdate string, to require a strtotime() in the comparison, a la:

if ($modifiedSince >= strtotime($lastModified))

But, I wonder why not just edit the get($key) function to deal with whatever format it might come across?

Anyway, I took a similar approach but adjusted the get($key) function to suit my needs. It works on my Digital Ocean test droplet running CentOS + Apache2 + PHP 5.5.14.

<?php

use \DateTime;
use Assetic\Asset\AssetInterface;
use Assetic\Cache\CacheInterface;
use Codesleeve\Sprockets\Interfaces\ClientCacheInterface;

class CustomClientCacheFilter extends \Codesleeve\AssetPipeline\Filters\ClientCacheFilter implements ClientCacheInterface
{
    public function get($key)                                                   
     {                                                                           
         $lastModified = $this->getLastTimeModified($key);                       
         $lmRFC7231    = @gmdate('D, d M Y H:i:s ', $lastModified).'GMT';        
         $modifiedSince = isset($_SERVER['HTTP_IF_MODIFIED_SINCE']) ? $_SERVER['HTTP_IF_MODIFIED_SINCE'] : 0;

         header('Last-Modified: '.$lmRFC7231.'GMT', true);                       
         header('Expires: -1', true);                                            
         header('Cache-Control: must-revalidate', true);                         

         if ($modifiedSince >= $lastModified)                                    
         {                                                                       
           header_remove("Expires");                                             
           header_remove("Cache-Control");                                       
           $expires = gmdate('D, d M Y H:i:s ', $lastModified + 31536000).'GMT'; // add 1 year
           header('HTTP/1.0 304 Not Modified');                                  
           header('Cache-Control: max-age=604799', true);                        
           header('Connection: Keep-Alive');                                     
           header('Etag: "'.$key.'"');                                           
           header("Expires: {$expires}", true);                                  
           header('Keep-Alive:timeout=5, max=97');                               
           exit;                                                                 
         }                                                                       

         return $this->cache->get($key);                                         
     }                                                                           


     private function getLastTimeModified($key)                                  
     {                                                                           
         return filemtime($this->asset->getSourceRoot() . '/' . $this->asset->getSourcePath());
     }
}                        

Maybe this will prove helpful?

[newbro]

I come across similar issue that the caching works with Chrome but for some reason always return 200 in IE. Debugging the ClientCacheFilter it seems the $_SERVER['HTTP_IF_MODIFIED_SINCE'] returns value in RFC 7231 format in IE, but returns timestamp format in Chrome.

I too believe that changing the get method would be better suit to ensure we are comparing two values of the same format.

[allella]

I see my dear friend @evantishuk came before me down this wretched path. Fortunately, he pays me to figure these things out.

His code above was actually not working correctly due to the first issue I explain below. Also, the Keep-Alive header should not be in there at all.

Problem A) In our LAMP environment (CentOS 6.6, PHP 5.5, mod_fcgid), and version of CodeSleeve's AssetPipeline\Filters\ClientCacheFilter (not sure which version because it's on dev-master) the base code returns a Last-Modified header of Thu, 01 Jan 1970 00:00:00 GMT. So, either that date, or something else causes the code to always send a Status 200 OK for CSS and JS files managed by CodeSleeve.

So, it's become necessary to add a CustomClientCacheFilter to extend the base filter class.

@matohavo explained where to put this custom class and how to enable it in Laravel in his Oct 8th comment above.

Here is the code that works for me in Firefox and Chrome (haven't tested IE or Opera yet).

Note that the exit() after the 304 header is absolutely necessary, or else the server will continue executing and send a status 200 and you'll wonder WTF that's happening.

Also, the getLastTimeModified() below is the same as the base class, but it must be required by the ClientCacheInterface, because it errors out without it.

<?php

use \DateTime;
use Assetic\Asset\AssetInterface;
use Assetic\Cache\CacheInterface;
use Codesleeve\Sprockets\Interfaces\ClientCacheInterface;

class CustomClientCacheFilter extends \Codesleeve\AssetPipeline\Filters\ClientCacheFilter implements ClientCacheInterface
{


    /**
     * If we make it here then we have a cached version of this asset
     * found in the underlying $cache driver. So we will check the
     * header HTTP_IF_MODIFIED_SINCE and if that is not less than
     * the last time we cached ($lastModified) then we will exit
     * with 304 header.
     *
     * @param  string $key
     * @return string
     */
    public function get($key)
    {
        $lastModified = $this->getLastTimeModified($key); // get the cached asset files last modified date
        $lmRFC7231    = @gmdate('D, d M Y H:i:s ', $lastModified).'GMT';  // convert that date to RFC 7231

          // see if the browser send a If-Modified-Since request header
        $modifiedSince = isset($_SERVER['HTTP_IF_MODIFIED_SINCE']) ? $_SERVER['HTTP_IF_MODIFIED_SINCE'] : 0;
          // calc expiration 4 months after last modified
        $expires = gmdate('D, d M Y H:i:s ', $lastModified + 10368000).'GMT';

          // if no If-Modified-Since request header was sent by the browser
          // OR if the browser's If-Modified-Since date and the asset's actual last modified don't match
        if ( $modifiedSince === 0 || ($lmRFC7231 != $modifiedSince) )
        {
              // cache validator headers (for helping invalidate/bust a changed asset)
            header('Last-Modified: ' . $lmRFC7231 . 'GMT', true);  // tell the browser the assets last modified date
            header('Etag: "' . $key . '"');  // set an ETag header

              // cache refresh info headers
            header('Cache-Control: max-age=10368000', true);  // tell browser to cache for up to 4 months
            header("Expires: " . $expires, true);  // set an Expires header

            return $this->cache->get($key);  // send the cached asset data
        }
        else
        {
            header("HTTP/1.1 304 Not Modified");  // send a not modified header to save bandwidth
            exit();  // this is absolutely necessary to force the 304 header to send immediately
        }
    }


    /**
     * Modified date format to work with hostmonster
     * @param  string $key
     * @return string
     */
    private function getLastTimeModified($key)
    {
        return filemtime($this->asset->getSourceRoot() . '/' . $this->asset->getSourcePath());
    }

}

Problem B) Running FastCGI or fcgid means the $_SERVER['HTTP_IF_MODIFIED_SINCE'] and $_SERVER['HTTP_IF_NONE_MATCH'] are not available to PHP. If you're running PHP as a CGI most likely you need to append the following to your .htaccess Rewrite rules, after all other RewriteRule, to make these values visible to PHP.

RewriteRule .* - [E=HTTP_IF_MODIFIED_SINCE:%{HTTP:If-Modified-Since}]
RewriteRule .* - [E=HTTP_IF_NONE_MATCH:%{HTTP:If-None-Match}]

I don't think this was necessary, but as mentioned above, it probably doesn't hurt to delete the asset cache files if you're having trouble. For Laravel, that looks like
rm -f ~/public_html/app/storage/cache/asset-pipeline/*