2009/07/22

still working on routing cache issue

I’m getting know routing cache problem now, but still struggling how to fix this. :( In my last post, I introduced lazy_routes_deserialize is the way to avoid this problem. Well, I believe some is true, but not everything. After I fixed my factories.yml configuration for sfPatterRouting, it seems that the performance of the application got a lot better, and I no longer across the error messages which I was frustrated very much. :)

However, I ran the application for a couple days, and found different problem. As the symfony blog mentioned , it generates too many routing cache files!!!!! It was more than 600000. 600000 in one directory is not very healthy. So, I changed my idea to use lazy_routes_deserialize. Instead, I decided to use sfAPCCache. The application was using APC anyway, so I thought put every routing cache in it. It worked fine, but I came across the exact same problem again, which is the size of routing cache gets huge! It ate almost 150M in 3 days, and it seems it is going to consume more in the near future. For now, I just run apache reload task one in a day, so it is working great. However, I don’t like the idea I’m using now. I have to find other strategies for this routing cache.

I think the base of this problem is my routing configuration. It gets so huge, and I need to find out better way to avoid it. The more specific URL you have, the bigger your routing cache gets. I understand it, but it seems the number it grows exponentially! :( It is not fair!

Is there any way to optimize my routing cache issue?

O.K, I found an answer. Do not use routing cache. Set sfNoCache instead.

2009/07/19

symfony routing cache and memory error

I was struggling with this trouble for almost 2 months!!!! My current project uses symfony as always, but I had a big problem with memory issue.

I am engaging this project for almost a year. This application was first created with symfony1.1, but now it is compatible with 1.2. Currently it uses many only 1.2 features such as routing, cache, and etc. I was very satisfied with symfony1.2, except this memory problem.

There were many out of memory error lines in my httpd error log.

  1. PHP Fatal error:  Allowed memory size of 100663296 bytes exhausted (tried to allocate 5926568 bytes) in /xxxxx/cache/frontend/prod/config/config_core_compile.yml.php on line 3366

Because of this error, I increased memory_limit to 96M in php.ini. However, it was not enough. Plus, because my server was a bit old and has only 512M, it always uses swap, and gets really bad sometimes. I was so frustrated with this problem, and googled this a lot. It is still a guess, and I don't know the solution I did was the exact one. BUT, It is working very good so far.

After I googled this issue, I found a couple similar topics in symfony forum as follows:

The first one was the problem with symfony1.2.4, and it said routing cache files size gets huge and takes too much memory to serialize it. At first, I did not know this post relates with my problem, because I was too lazy to look inside of line 3366 in config_core_compile.yml.php. After checking line 3366, I found it was the same problem. Too much memory consumption exceeded allowed memory size. One of this fix is using SQLite to handle this cache, but in my understanding, symfony1.2.7 provides different way to solve this problem, which you can find symfony 1.2.7 - more power.

  1. all:
  2.   routing:
  3.     class: sfPatternRouting
  4.     param:
  5.       generate_shortest_url:            true
  6.       extra_parameters_as_query_string: true
  7.       lazy_routes_deserialize:          true
  8.       lookup_cache_dedicated_keys:      true

I checked how this fix affects. This fix generate many routing cache files in cache/app_name/prod/config/routing/. The file size was not huge, but number of files is huge, as the symfony blog says. I might need to use different storage for this cache handling, but now the memory error is disappeared. I am very happy with it. :)

Man, i should have noticed symfony1.2.7 fix. :(

BUT, it did not fix, but it is working now, The answer is "Do not use routing cache. Set sfNoCache instead."

Bloglines feedburner