Before I bought U-xer, I asked Zekeriya if it would be possible to run kind of click bot on the Ipad. He told me, that it would be possible, if the Ipad is connected to the Macbook. Unfortunately I found no helping source, how to start and do it.
So quick overview what should be done:
Setting: Ipad screen is open and on my left of my Macbook. I theroretically can move with my mouse over to the Ipad and steer it via the Macbook.
What to do:
it should open an app. → how can this be achieved, that it opens the app on the Ipad? Can there be a “database” trained with screenshots, where for example the App Icon is trained, so it scans the screen and then click the App Icon when found?
Here I struggle the first time. How can I make U-xer to do somthing on the Ipad Screen?
same then in the app: it should scan the screen and should do several clicking tasks according a rundown. Like “click first this icon” (pre trained picture?), on the popup: “click this Icon 5x” (also pre-trained picture?) and so on.
So basically I screenshot through the app and create a database with little thumbnails and give them like a constant value. Like I have created a thumbnail of a “coin” icon and name this “coin”. I have a thumbnail of a “bill” and name it “bill”, I have a thumbnail of a “boat” and name it “boat” etc. So when I write the scenario, I tell u-xer: "When screen is loaded, click on “coin” and it opens the popup. Then I say: “click on bill” and it clicks on it. Close the popup with the “X” and then click on “boat” and so on.
So the first step is to download and install the agent.
There are bunch of steps to follow while configuring IOS automation. It is pretty new and we are still preparing documentation about it. In the mean time, you can give it a try to do it by yourself following the instructions on the agent.
Go to getting started section and see how things work in U-xer. It will be exactly the same for Ipad, the only difference will be the first setup.
When setup is ready, then clicking icons or texts are easy.
I tried on my own, but it was somehow not connecting with the Ipad. On the agent, everything was loaded green, the Ipad was connected as well, however it seems not working. Until a IOS Version is out, I might try the Android way. Since the scenario will be the same, I could already try that first.
However, is it possible to train a picture library (out of a pdf or whatever file), which is “preloaded” when starting the scenario, so it searches not only the text, but for pictures? How how can this be done? I didn’t found anything like that in the tutorial scenarios.
About Ipad, what is you OS version? Did you click Iphone on the agent? Did you install Xcode? Let’s wait Until we complete the documentation about this.
About using image on the PDF, you can use screen scraper for that. Technically speaking, it is not called training but more like image search.
IPadOS-Version is 17.5.1.
I also still have an IPadOS-Version 17.4.1 to test it, if it would work there.
I will also try the Screenshot Scraper, as soon as it is working on either system. Still trying to find a MacOS App to connect to Android not only as mirror, but also as connection to interact with it.
no, so far I had no time to check the Android version, since my native devices are all Apple. However, I guess the problem to solve is how to deploy the Uxer-Runner app on the IOS devices. I tried now 2 Ipads, both with the same result.
All requirements are green (Xcode, Command Line, Xcode Project). It detects also each device (Select Device: Number ok). But when clicking “Start Automation” it tries to deploy the Uxer-Runner on the device, “please wait” and then it writes “Unable to start automation on iPhone. Please ensure iPhone is connected via USB (it is) and the app Uxer-Runner is installed and running”. At this point it fails, since it does not deploys the Uxer-Runner on the devices.
Is there a way to install the Runner from an external address, not via the Settings Menu?
Thanks for the guide, but I still was not able to connect to the Ipad. I tried 2 different ones, I followed all the steps for IOS Automation for Mac, but it was not able to deploy something to the Ipads, nor a VPN connection was created on the Ipads. It just hangs up and somewhen abort. I think it might be something from X-Code not set right. However, it connects to the Ipad, then it switches to kind of reloading Ipad and so on. The test itself and completed.
There are several warnings and caution messages in X-Code:
Multiple commands produce ‘/Users/alexander/Library/Developer/Xcode/DerivedData/WebDriverAgent-axnvyzwyhdqdsrbazjgjwviesiux/Build/Products/Debug-iphoneos/Uxer-Runner.app/PlugIns/Uxer.xctest/Info.plist’
Target ‘Uxer’ (project ‘WebDriverAgent’) has copy command from ‘/var/folders/tv/s4p1f0fj6ml1ggvb678cqkc00000gn/T/UxerIOS/uxeriosagent/WebDriverAgentRunner/Info.plist’ to ‘/Users/alexander/Library/Developer/Xcode/DerivedData/WebDriverAgent-axnvyzwyhdqdsrbazjgjwviesiux/Build/Products/Debug-iphoneos/Uxer-Runner.app/PlugIns/Uxer.xctest/Info.plist’
Target ‘Uxer’ (project ‘WebDriverAgent’) has process command with output ‘/Users/alexander/Library/Developer/Xcode/DerivedData/WebDriverAgent-axnvyzwyhdqdsrbazjgjwviesiux/Build/Products/Debug-iphoneos/Uxer-Runner.app/PlugIns/Uxer.xctest/Info.plist’
Multiple commands produce ‘/Users/alexander/Library/Developer/Xcode/DerivedData/WebDriverAgent-axnvyzwyhdqdsrbazjgjwviesiux/Build/Products/Debug-iphoneos/WebDriverAgentLib.framework/Info.plist’
Target ‘WebDriverAgentLib’ (project ‘WebDriverAgent’) has copy command from ‘/var/folders/tv/s4p1f0fj6ml1ggvb678cqkc00000gn/T/UxerIOS/uxeriosagent/WebDriverAgentRunner/Info.plist’ to ‘/Users/alexander/Library/Developer/Xcode/DerivedData/WebDriverAgent-axnvyzwyhdqdsrbazjgjwviesiux/Build/Products/Debug-iphoneos/WebDriverAgentLib.framework/Info.plist’
Target ‘WebDriverAgentLib’ (project ‘WebDriverAgent’) has process command with output ‘/Users/alexander/Library/Developer/Xcode/DerivedData/WebDriverAgent-axnvyzwyhdqdsrbazjgjwviesiux/Build/Products/Debug-iphoneos/WebDriverAgentLib.framework/Info.plist’
