You're correct. You have to fetch the data from the database and use something along the line of:

  function initialize() {

    var myLatlng = new google.maps.LatLng(<?php echo $lat; ?>,<?php echo $long; ?>);

    var myOptions = {

      zoom: 4,

      center: myLatlng,

      mapTypeId: google.maps.MapTypeId.ROADMAP

    }

    var map = new google.maps.Map(document.getElementById(map_canvas), myOptions);



    var marker = new google.maps.Marker({

        position: myLatlng, 

        map: map,

        title:Hello World!

    });   

  }

In a < script type=text/javascript>-tag on the page.
The above is taken from http://code.google.com/apis/maps/documentation/javascript/examples/marker-simple.html, so you'll have to adapt it to your needs.