@abhay First of all, I am a total infrastructure noob, so I might be mistaken…
Having said that, I am missing the following in your virtual host definition:
<Directory /var/www/>
Options Indexes FollowSymLinks
AllowOverride All
Require all granted
</Directory>
When I comment out the lines containing ‘AllowOverride All’ and ‘Require all granted’ I can only access the home page. When accessing a blog item I get the exact same error as on your site.
Do I know what that all does? No, I don’t… I rather read spaghetti code…