Nowadays Xorg is doing a great job automatically recognizing and configuring available hardware. In case of monitors this procedure relies on the so called EDID record which monitors provide on demand over standard interfaces. Thus, if the information provided by the monitor is corrupted (parity check), it is simply disregarded to prevent damage to your equipment and a set of very conservative modes is probed to find a working one. Luckily the resolution can be fixed even for the already running X session.

I have LMDE 17 with Mate later enriched with xfce. At some point I thought the resolution and/or dead DIV-D ports on my docking station are caused by the open source radeon driver, so I tried to use the proprietary fglrx driver first from the LMDE distro. And as it turned out that my new card was not supported by the driver, I uploaded and brut force installed the driver provided by AMD (https://help.ubuntu.com/community/BinaryDriverHowto/AMD) (bad idea, but it works even this way).

If automatic hardware detection did not work to your satisfaction, it means you need to configure your X session manually. This page here http://wiki.cchtml.com/index.php/Main_Page has plenty of useful instructions how to get a basic setup.

At this moment you should have your X session up and running, but your monitor has a poor resolution and you can not change it to the expected one.

$ gtf 1920 1200 60

  # 1920x1200 @ 60.00 Hz (GTF) hsync: 74.52 kHz; pclk: 193.16 MHz
  Modeline "1920x1200_60.00"  193.16  1920 2048 2256 2592  1200 1201 1204 1242  -HSync +Vsync

$ xrandr --newmode "1920x1200_60.00"  193.16  1920 2048 2256 2592  1200 1201 1204 1242  -HSync +Vsync
$ xrandr --addmode VGA1 1920x1200_60.00
$ xrandr --output VGA1 --mode 1920x1200_60.00

results in an error message, and does not make your monitor resolution and life better. Note here "VGA1" is the name of my output, you should figure out the proper name for your case examining the output of

$ xrandr -q

The reader should provide his desired horizontal, vertical resolutions and the refresh rate to gtf command.

At this point you should check your monitor's EDID information. The easiest way to do this is to install the read-edid package, use your favorite way to do it, I myself prefer Synaptic package manager. If you can get a clean EDID record as for example the one reported here http://hotcashew.com/2013/08/fixing-invalid-edid-in-linux-wit-fglrx/, you should stop reading this tutorial and look what is wrong with your system elsewhere. Otherwise, we need a correct version of EDID record for your monitor. There are several methods to get it, starting from "ack monitor vendor" ending with "getting a hand on a same model monitor" as in the example above. I had to boot into Windows, install Phoenix EDID tool (together with 3-5 other creepy software packages), almost as described at the end of the page here: http://ubuntuforums.org/showthread.php?t=1938447&page=4&p=11938884#post11938884. Phoenix allows you to save EDID information into a edid_file.raw, which you should put somewhere, you know how to get to when you are booted into your linux. When you are rebooted into linux you can check the integrity of your new EDID file like this:

$ sudo parse-edid edid_file.raw
parse-edid: parse-edid version 2.0.0
parse-edid: EDID checksum passed.

    # EDID version 1 revision 3
Section "Monitor"
    # Block type: 2:0 3:fd
    # Block type: 2:0 3:fc
    Identifier "EA241WM"
    VendorName "NEC"
    ModelName "EA241WM"
    # Block type: 2:0 3:fd
    HorizSync 26-82
    VertRefresh 50-76
    # Max dot clock (video bandwidth) 170 MHz
    # Block type: 2:0 3:fc
    # Block type: 2:0 3:ff
    # DPMS capabilities: Active off:yes  Suspend:yes  Standby:yes

    Mode     "1920x1200"    # vfreq 59.950Hz, hfreq 74.038kHz
        DotClock    154.000000
        HTimings    1920 1968 2000 2080
        VTimings    1200 1203 1209 1235
        Flags    "-HSync" "+VSync"
    EndMode
    # Block type: 2:0 3:fd
    # Block type: 2:0 3:fc
    # Block type: 2:0 3:ff
EndSection
 

Now you can construct a new "Modeline" from the Mode section of EDID:

Modeline "1920x1200" 154.00 1920 1968 2000 2080 1200 1203 1209 1235 -hsync +vsync

in my case, and use this line in the Monitor section of your xorg.conf file, as described here https://delightlylinux.wordpress.com/2014/01/13/edid-revisted/.

But before you do it, check everything manually
$ xrandr --newmode "1920x1200"  154.00 1920 1968 2000 2080 1200 1203 1209 1235 -hsync +vsync
$ xrandr --addmode VGA1 1920x1200
$ xrandr --output VGA1 --mode 1920x1200

I hope it helped. I do not want to go into details on how to make the changes permanent. :-)