Adding Javascript to The "Write" Step

❓ Help and Questions
1

When I add javascript that contains ‘\n’ to the “Write” step, the app seems to strip the ‘\n’ part out when writing it?

OpenBrowser Chrome “https://www.google.com/”
Write “keyword”
SendKeys ENTER
SendKeys CMD+OPTION+VK_J
Click Text “Console”
Write “(async function(){const delay=ms=>new Promise(resolve=>setTimeout(resolve,ms));async function scrollToEnd(){let lastScrollTop=document.documentElement.scrollTop;while(true){window.scrollTo(0,document.documentElement.scrollHeight);await delay(1000);let currentScrollTop=document.documentElement.scrollTop;if(currentScrollTop===lastScrollTop){break}lastScrollTop=currentScrollTop}}await scrollToEnd();const links=Array.from(document.querySelectorAll(‘a’)).filter(link=>link.querySelector(‘h3’));const hrefs=links.map(link=>link.href).join(‘\n’);const textarea=document.createElement(‘textarea’);textarea.value=hrefs;document.body.appendChild(textarea);textarea.select();document.execCommand(‘copy’);document.body.removeChild(textarea);console.log(‘Links copied to clipboard’);})();”
SendKeys RETURN

2

Can you add "" to escape from special characters?

This worked for me:
Write “test \n test2”

image
image1920×1014 176 KB

3

Ok, yes. Adding the 2 lines before the n instead of \n worked. It goes to google and searches a keyword and then saves all the links from the google results to clipboard.

OpenBrowser Chrome "https://www.google.com/"
Write "keyword"
SendKeys ENTER
SendKeys CMD+OPTION+VK_J
Write "(async function(){const delay=ms=>new Promise(resolve=>setTimeout(resolve,ms));async function scrollToEnd(){let lastScrollTop=document.documentElement.scrollTop;while(true){window.scrollTo(0,document.documentElement.scrollHeight);await delay(1000);let currentScrollTop=document.documentElement.scrollTop;if(currentScrollTop===lastScrollTop){break}lastScrollTop=currentScrollTop}}await scrollToEnd();const links=Array.from(document.querySelectorAll('a')).filter(link=>link.querySelector('h3'));const hrefs=links.map(link=>link.href).join('\\n');const textarea=document.createElement('textarea');textarea.value=hrefs;document.body.appendChild(textarea);textarea.select();document.execCommand('copy');document.body.removeChild(textarea);console.log('Links copied to clipboard');})();"
SendKeys RETURN
1 Like
4

This is absolutely fantastic!

Deserves to be shared in use cases!

2 Likes
5

Just updating this in case it’s useful to anyone. It scrapes the links from the first 5 pages of google results. Each results page it saves/appends the links to chrome local storage and then at the end of the 5 pages it saves the links from storage to the clipboard.

OpenBrowser Chrome "https://www.google.com/"
IsVisible Text "Google Search"
Write "keyword"
SendKeys ENTER
SendKeys CMD+OPTION+VK_J
Write "(async function(){localStorage.removeItem('links');const delay=ms=>new Promise(resolve=>setTimeout(resolve,ms));async function scrollToEnd(){let lastScrollTop=document.documentElement.scrollTop;while(true){window.scrollTo(0,document.documentElement.scrollHeight);await delay(1000);let currentScrollTop=document.documentElement.scrollTop;if(currentScrollTop===lastScrollTop){break}lastScrollTop=currentScrollTop}}await scrollToEnd();const links=Array.from(document.querySelectorAll('a')).filter(link=>link.querySelector('h3'));const hrefs=links.map(link=>link.href).join('\\n');let existingHrefs=localStorage.getItem('links')||'';existingHrefs+=existingHrefs?'\\n'+hrefs:hrefs;localStorage.setItem('links',existingHrefs);console.log('Links appended to local storage');})();"
SendKeys  ENTER
Click Text "Next"
SendKeys CMD+OPTION+VK_J
SendKeys CMD+OPTION+VK_J
Write "(async function(){const delay=ms=>new Promise(resolve=>setTimeout(resolve,ms));async function scrollToEnd(){let lastScrollTop=document.documentElement.scrollTop;while(true){window.scrollTo(0,document.documentElement.scrollHeight);await delay(1000);let currentScrollTop=document.documentElement.scrollTop;if(currentScrollTop===lastScrollTop){break}lastScrollTop=currentScrollTop}}await scrollToEnd();const links=Array.from(document.querySelectorAll('a')).filter(link=>link.querySelector('h3'));const hrefs=links.map(link=>link.href).join('\\n');let existingHrefs=localStorage.getItem('links')||'';existingHrefs+=existingHrefs?'\\n'+hrefs:hrefs;localStorage.setItem('links',existingHrefs);console.log('Links appended to local storage');})();"
SendKeys  ENTER
Click Text "Next"
SendKeys CMD+OPTION+VK_J
SendKeys CMD+OPTION+VK_J
Write "(async function(){const delay=ms=>new Promise(resolve=>setTimeout(resolve,ms));async function scrollToEnd(){let lastScrollTop=document.documentElement.scrollTop;while(true){window.scrollTo(0,document.documentElement.scrollHeight);await delay(1000);let currentScrollTop=document.documentElement.scrollTop;if(currentScrollTop===lastScrollTop){break}lastScrollTop=currentScrollTop}}await scrollToEnd();const links=Array.from(document.querySelectorAll('a')).filter(link=>link.querySelector('h3'));const hrefs=links.map(link=>link.href).join('\\n');let existingHrefs=localStorage.getItem('links')||'';existingHrefs+=existingHrefs?'\\n'+hrefs:hrefs;localStorage.setItem('links',existingHrefs);console.log('Links appended to local storage');})();"
SendKeys  ENTER
Click Text "Next"
SendKeys CMD+OPTION+VK_J
SendKeys CMD+OPTION+VK_J
Write "(async function(){const delay=ms=>new Promise(resolve=>setTimeout(resolve,ms));async function scrollToEnd(){let lastScrollTop=document.documentElement.scrollTop;while(true){window.scrollTo(0,document.documentElement.scrollHeight);await delay(1000);let currentScrollTop=document.documentElement.scrollTop;if(currentScrollTop===lastScrollTop){break}lastScrollTop=currentScrollTop}}await scrollToEnd();const links=Array.from(document.querySelectorAll('a')).filter(link=>link.querySelector('h3'));const hrefs=links.map(link=>link.href).join('\\n');let existingHrefs=localStorage.getItem('links')||'';existingHrefs+=existingHrefs?'\\n'+hrefs:hrefs;localStorage.setItem('links',existingHrefs);console.log('Links appended to local storage');})();"
SendKeys  ENTER
Click Text "Next"
SendKeys CMD+OPTION+VK_J
SendKeys CMD+OPTION+VK_J
Write "(async function(){const delay=ms=>new Promise(resolve=>setTimeout(resolve,ms));async function scrollToEnd(){let lastScrollTop=document.documentElement.scrollTop;while(true){window.scrollTo(0,document.documentElement.scrollHeight);await delay(1000);let currentScrollTop=document.documentElement.scrollTop;if(currentScrollTop===lastScrollTop){break}lastScrollTop=currentScrollTop}}await scrollToEnd();const links=Array.from(document.querySelectorAll('a')).filter(link=>link.querySelector('h3'));const hrefs=links.map(link=>link.href).join('\\n');let existingHrefs=localStorage.getItem('links')||'';existingHrefs+=existingHrefs?'\\n'+hrefs:hrefs;localStorage.setItem('links',existingHrefs);console.log('Links appended to local storage');})();"
SendKeys  ENTER
Write "const hrefs=localStorage.getItem('links')||'';const textarea=document.createElement('textarea');textarea.value=hrefs;document.body.appendChild(textarea);textarea.select();document.execCommand('copy');document.body.removeChild(textarea);console.log('Links copied to clipboard');"
SendKeys ENTER
2 Likes
6

Thanks a lot Shane for sharing :slight_smile:

1 Like
7

Great work. I have tried using code to get this to send to the scrapped links to a google sheet via an appscript but I cant get it working.

This is what was added to the last code block to replace - const hrefs=localStorage.getItem(‘links’)||‘’;const textarea=document.createElement(‘textarea’);textarea.value=hrefs;document.body.appendChild(textarea);textarea.select();document.execCommand(‘copy’);document.body.removeChild(textarea);console.log(‘Links copied to clipboard’);

The new code I am trying to use:

const hrefs = localStorage.getItem(‘links’) || ‘’;

if (hrefs) {
const url = ‘YOUR_GOOGLE_APPS_SCRIPT_WEB_APP_URL’;

fetch(url, {
method: ‘POST’,
headers: {
‘Content-Type’: ‘application/x-www-form-urlencoded’
},
body: new URLSearchParams({ data: hrefs })
})
.then(response => response.text())
.then(result => {
console.log(‘Data sent to Google Sheets:’, result);
})
.catch(error => {
console.error(‘Error:’, error);
});
}

and this is the appscript:

function doPost(e) {
var sheet = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet();
var data = e.parameter.data;

// Assuming the data is a comma-separated string of links
var links = data.split(‘,’);

// Insert each link into a new row
for (var i = 0; i < links.length; i++) {
sheet.appendRow([links[i]]);
}

return ContentService.createTextOutput(‘Success’);
}

Any thoughts how to do this as it would be an amazing automation

8

This topic started get more complicated then I can handle :slight_smile:

@rapidresultsmarketin ,

Were you able to run this script work manually?
If it worked when you run it manually but didn’t work when you use U-xer, we will focus on what U-xer write on the console. Due to some special characters, we may need to update the script.

9

I’m not sure how you would get that to work rn but I wonder if you could have U-xer open google sheets in a new tab after all the links have been scraped and U-xer paste the links into the table? Each link should automatically go to a new row.