Sometimes when creating e2e tests, and this becomes even more important as we move into the world of AI, there is a gap between actual user experience and what the AI sees. Recently I had an issue where I was creating tests with help from AI.

The test needed to prove some new functionality. A user would create a new facility, and if there was an existing facility that was similar, we would show the user the existing facilities, and if one was the actual facility the user needed, they could click select, and it would populate the parent facility select field. Simple enough!

So we wrote the functionality, and I tried to test. After much back and forth, here is the takeaways.

The AI makes assumptions.

The first hurdle was passing tests but it wasn’t working for me. Ended up finding out that the AI had written tests that used a testing interface or sandbox I had set up earlier to get things working. Because this was testing the functionality from a root page and not a modal, it was passing. However we needed it to work from a modal. After waaaay too long, we updated to test against the modal. PASS! but hang on, still not working right for me!

This is both one of my fave parts of programming and one of my least faves depending on how rushed I am. It requires the cause and effect method. If I do this, what should happen? What about that? I decided first of all I needed to prove I was clicking the button in a provable way. Sometimes AI doesn’t trust me 🤣

I got AI to write a js script to paste in:

(function(){
  if(document.getElementById('click-log')) return;
  var d=document.createElement('pre');
  d.id='click-log';
  d.style.cssText='position:fixed;top:0;left:0;z-index:99999;background:#111;color:#0f0;padding:10px;max-height:300px;overflow-y:auto;font-size:11px;width:500px;opacity:0.9;pointer-events:none;font-family:monospace';
  document.body.appendChild(d);
  var L=[];
  function log(m){ var t=new Date().toISOString().substring(11,23); L.push(t+' '+m); d.textContent=L.join('\n'); console.log('CLICK-LOG|'+t+'|'+m); }
  document.addEventListener('click',function(e){
    var el=e.target;
    log('CLICK: <'+el.tagName.toLowerCase()+'>'+
      ' id='+(el.id||'-')+
      ' cls='+(el.className||'').substring(0,25)+
      ' txt="'+(el.textContent||'').substring(0,25).replace(/\s+/g,' ')+'"'+
      (el.getAttribute('data-facility-id')?' data-fid='+el.getAttribute('data-facility-id'):'')+
      (el.getAttribute('data-modal-id')?' data-mid='+el.getAttribute('data-modal-id'):'')
    );
  },true);
  document.addEventListener('click',function(e){
    log('CLICK-BUBBLE: <'+e.target.tagName.toLowerCase()+'> id='+(e.target.id||'-'));
  },false);
  log('READY');
})();

then I pasted it in the browser, and did the actions. I noticed that only the second click registered!

If I clicked out of the text box, and then select, it worked. But going straight from the text box to clicking select, it failed.

In the end it was because the app was generating the select buttons on textbox change. Once it was changed to key down, it worked!