The file browser Caja of the MATE desktop does not start when added to Docky. A Google search tells us that this problem exists since at least 2010. A recent installation of Linux Mint 18.1 with MATE desktop 1.16.1 and Docky 2.2.1.1-1 still has the same problem and suggestions found on several forums did not solve the problem either.

So I decided to do some experimenting with the desktop-file. I added StartupWMClass=caja and added a closing dot to the command: Exec=caja --browser . I also set Hidden=false. See the notes below. These adjustments seem to do the trick. This is what the adjusted dektop-file looks like:

#!/usr/bin/env xdg-open
[Desktop Entry]
Version=1.0
Type=Application
Terminal=false
Exec=caja --browser .
Name=Caja
Hidden=false
Icon=file-manager
StartupWMClass=caja

Notes

1. Instead of changing the desktop file in "/usr/share/applications" I created a new desktop file in my home folder under ".local/share/applications", set its permission to execute and dragged that file to Docky.

2. The entry "Hidden=false" ensures that no second Caja-icon appears on Docky. The downside is that you can launch only one instance of Caja from Docky. If you set "Hidden=true" then a second Caja icon appears on the right of Docky. You can then launch multiple instances and show/hide all those instances simply by toggling that second icon.