IMHO, the best RAM metric available in Activity Monitor is Page Outs. RAM demand fluctuates too rapidly to be reflected meaningfully in either Free or Inactive RAM.
But Page Outs happen because of a RAM shortage, however transient. Until the Page Out occurs, the task that needed the memory is stalled, and this probably results in delay for you. It also will result in Page Ins at some future time, meaning more delay.
High rates of Page Outs result in sluggishness, although SSDs can tolerate maybe ten times the paging rate that HDDs can sustain.
You'll have to decide just how much paging affects your working and how sensitive you are to slowdowns.