Diff
Modified: trunk/Source/WebKit/ChangeLog (230829 => 230830)
--- trunk/Source/WebKit/ChangeLog 2018-04-20 01:00:40 UTC (rev 230829)
+++ trunk/Source/WebKit/ChangeLog 2018-04-20 01:45:21 UTC (rev 230830)
@@ -1,3 +1,129 @@
+2018-04-19 Brian Burg <bb...@apple.com>
+
+ Web Automation: add support for mouse/keyboard interaction sequences
+ https://bugs.webkit.org/show_bug.cgi?id=184603
+ <rdar://problem/39421839>
+
+ Reviewed by Timothy Hatcher.
+
+ Add new protocol methods and WebKit support for implementing the W3C Actions API.
+ This is a generic command for sending low-level mouse, key, and touch events to
+ test page behavior when performing drag-and-drop, gestures, or specific keydown/keyups.
+
+ To implement this functionality, this patch adds SimulatedInputDispatcher, a class
+ for dispatching input events asynchronously. Similar to the WebDriver specification,
+ this is designed as a keyframing API. Callers set up several "input sources" such
+ as keyboard or mouse, and then specify the states of that input source over time. The
+ dispatcher calculates diffs between the previous and current keyframes and generates
+ the appropriate events that would happen if a user caused the state transition.
+
+ For example, if a mouse input source's state changes, the dispatcher sends synthetic mousemove,
+ mousedown, or mouseup events depending on the pre- and post-state. This is uninteresting
+ and overcomplicated for simple key and mouse presses, but it's really designed with an
+ eye towards supporting mousemove interpolation and touch event interpolation in later patches.
+
+ The strategy for dispatching events depends on the input source type; right now, these
+ map directly to the existing platformSimulate{Mouse, Keyboard}Interaction methods. In
+ the future, the dispatch strategy may be elaborated for interpolated mousemove events.
+
+ This patch depends on support added in bug 184462.
+
+ No tests yet. When this command is hooked up to a driver, the code will be exercised by
+ W3C actions test suite, which is fairly complex at this point relative to end-user code.
+
+ * UIProcess/Automation/Automation.json: Add new types and commands.
+
+ * UIProcess/Automation/SimulatedInputDispatcher.h: Added.
+ (WebKit::SimulatedInputSourceState::emptyState):
+ (WebKit::SimulatedInputSource::create):
+ (WebKit::SimulatedInputSource::SimulatedInputSource):
+ (WebKit::SimulatedInputDispatcher::Client::~Client):
+ Add structs for input source, source state, and keyframe.
+ The dispatcher's client interface is implemented by WebAutomationSession.
+
+ (WebKit::SimulatedInputDispatcher::create):
+ * UIProcess/Automation/SimulatedInputDispatcher.cpp: Added.
+ (WebKit::SimulatedInputKeyFrame::SimulatedInputKeyFrame):
+ (WebKit::SimulatedInputKeyFrame::maximumDuration const):
+ (WebKit::SimulatedInputKeyFrame::keyFrameFromStateOfInputSources):
+ (WebKit::SimulatedInputKeyFrame::keyFrameToResetInputSources):
+ (WebKit::SimulatedInputDispatcher::SimulatedInputDispatcher):
+ (WebKit::SimulatedInputDispatcher::~SimulatedInputDispatcher):
+ (WebKit::SimulatedInputDispatcher::isActive const):
+ (WebKit::SimulatedInputDispatcher::keyFrameTransitionDurationTimerFired):
+ (WebKit::SimulatedInputDispatcher::isKeyFrameTransitionComplete const):
+ (WebKit::SimulatedInputDispatcher::transitionToNextKeyFrame):
+ (WebKit::SimulatedInputDispatcher::transitionToNextInputSourceState):
+ (WebKit::SimulatedInputDispatcher::transitionBetweenKeyFrames):
+ (WebKit::SimulatedInputDispatcher::transitionInputSourceToState):
+ (WebKit::SimulatedInputDispatcher::run):
+ (WebKit::SimulatedInputDispatcher::cancel):
+ (WebKit::SimulatedInputDispatcher::finishDispatching):
+ The dispatcher handles one interaction at a time. The interaction is described
+ by an array of keyframes, and each keyframe has an array of states. The dispatcher
+ transitions between keyframes by sequentially and asynchronously emitting events
+ that cause each input source state to transition as desired. Keyframe transitions
+ are additionally gated by a "maximum duration" timer. Each step that the dispatcher
+ executes is asynchronous, so the dispatcher keeps most state in members and uses
+ error argument lambdas as completion handlers for various async things.
+
+ * UIProcess/Automation/WebAutomationSession.h:
+ * UIProcess/Automation/WebAutomationSession.cpp:
+ (WebKit::WebAutomationSession::WebAutomationSession):
+ (WebKit::WebAutomationSession::inputDispatcherForPage):
+ (WebKit::WebAutomationSession::inputSourceForType const):
+ Add canonical input sources that are used to keep track of state across
+ interaction sequences.
+
+ (WebKit::WebAutomationSession::isSimulatingUserInteraction const):
+ (WebKit::WebAutomationSession::mouseEventsFlushedForPage):
+ (WebKit::WebAutomationSession::keyboardEventsFlushedForPage):
+ Remove m_simulatingUserInteraction since it can be computed based on other members.
+
+ (WebKit::WebAutomationSession::willClosePage):
+ If the page is being torn down, stop the dispatcher if needed and cancel any
+ callbacks waiting for mouse/key events to be retired.
+
+ (WebKit::WebAutomationSession::simulateMouseInteraction):
+ (WebKit::WebAutomationSession::simulateKeyboardInteraction):
+ Add easy-to-use async methods for simulating mouse and key events. These are
+ hooked up to SimulatedInputDispatcher using async completion handlers.
+
+ (WebKit::protocolMouseButtonToWebMouseEventButton):
+ (WebKit::WebAutomationSession::performMouseInteraction):
+ (WebKit::WebAutomationSession::performKeyboardInteractions):
+ Adjust some naming.
+
+ (WebKit::simulatedInputSourceTypeFromProtocolSourceType):
+ (WebKit::WebAutomationSession::performInteractionSequence):
+ (WebKit::WebAutomationSession::cancelInteractionSequence):
+ Add command handlers for the new action commands in Automation protocol.
+
+ * UIProcess/Automation/gtk/WebAutomationSessionGtk.cpp:
+ (WebKit::mouseButtonToGdkButton):
+ (WebKit::WebAutomationSession::platformSimulateMouseInteraction):
+ (WebKit::WebAutomationSession::platformSimulateKeyboardInteraction):
+ (WebKit::WebAutomationSession::platformSimulateKeyStroke): Deleted.
+ * UIProcess/Automation/ios/WebAutomationSessionIOS.mm:
+ (WebKit::WebAutomationSession::platformSimulateKeyboardInteraction):
+ (WebKit::WebAutomationSession::platformSimulateKeyStroke): Deleted.
+ Rename the keyboard platform method to match the naming of the mouse platform method.
+ Take advantage of the 'using' alias to make the tedious switches easier to read.
+
+ * UIProcess/Automation/mac/WebAutomationSessionMac.mm:
+ (WebKit::WebAutomationSession::platformSimulateMouseInteraction):
+ (WebKit::virtualKeyHasStickyModifier):
+ (WebKit::keyCodeForVirtualKey):
+ (WebKit::eventModifierFlagsForVirtualKey):
+ (WebKit::WebAutomationSession::platformSimulateKeyboardInteraction):
+ (WebKit::WebAutomationSession::platformSimulateKeySequence):
+ (WebKit::keyHasStickyModifier): Deleted.
+ (WebKit::WebAutomationSession::platformSimulateKeyStroke): Deleted.
+ Allow the keyboard simulation method to take a virtual key and unichar to better
+ match how this is used by the Perform Actions command and its machinery.
+
+ * WebKit.xcodeproj/project.pbxproj:
+
2018-04-19 Jiewen Tan <jiewen_...@apple.com>
Remove access to keychain from the WebContent process
Modified: trunk/Source/WebKit/UIProcess/Automation/Automation.json (230829 => 230830)
--- trunk/Source/WebKit/UIProcess/Automation/Automation.json 2018-04-20 01:00:40 UTC (rev 230829)
+++ trunk/Source/WebKit/UIProcess/Automation/Automation.json 2018-04-20 01:45:21 UTC (rev 230830)
@@ -72,7 +72,8 @@
"ElementNotInteractable",
"ElementNotSelectable",
"ScreenshotError",
- "UnexpectedAlertOpen"
+ "UnexpectedAlertOpen",
+ "TargetOutOfBounds"
]
},
{
@@ -244,6 +245,46 @@
{ "name": "permission", "$ref": "SessionPermission" },
{ "name": "value", "type": "boolean" }
]
+ },
+ {
+ "id": "InputSourceType",
+ "type": "string",
+ "description": "The type of an input source. This determines which state properties are applicable to the input source. The 'Null' input source type has no properties, only a duration. This can extend the effective wait for a tick.",
+ "enum": [
+ "Null",
+ "Mouse",
+ "Keyboard",
+ "Touch"
+ ]
+ },
+ {
+ "id": "InputSource",
+ "type": "object",
+ "properties": [
+ { "name": "sourceId", "type": "string", "description": "A unique identifier for this input source." },
+ { "name": "sourceType", "$ref": "InputSourceType", "description": "The type of this input source. This affects what input source state fields are valid for this input source." }
+ ]
+ },
+ {
+ "id": "InteractionStep",
+ "description": "A set of input source states to which all applicable input sources must transition before further steps may be processed. If an input source does not have a corresponding state in a step, its state assumed to be 'released' for that step.",
+ "type": "object",
+ "properties": [
+ { "name": "states", "type": "array", "items": { "$ref": "InputSourceState" }, "optional": true, "description": "A list of new input states for input sources that must transition during this step. Source state transitions that cannot be performed concurrently (i.e., touch gestures) are performed sequentially in the order they are listed." }
+ ]
+ },
+ {
+ "id": "InputSourceState",
+ "type": "object",
+ "description": "A new state for a specific input source. All state-related fields are optional and must be applicable to the InputSource referenced by 'sourceId'. If no state-related fields are specified, the state is assumed to remain the same as in the previous step (i.e., 'sustained').",
+ "properties": [
+ { "name": "sourceId", "type": "string", "description": "The input source whose state is described by this object." },
+ { "name": "pressedCharKey", "type": "string", "optional": true, "description": "For 'keyboard' input sources, specifies a character key that has 'pressed' state. Unmentioned character keys are assumed to have a 'released' state." },
+ { "name": "pressedVirtualKey", "$ref": "VirtualKey", "optional": true, "description": "For 'keyboard' input sources, specifies a virtual key that has a 'pressed' state. Unmentioned virtual keys are assumed to have a 'released' state." },
+ { "name": "pressedButton", "$ref": "MouseButton", "optional": true, "description": "For 'mouse' input sources, specifies which mouse button has a 'pressed' state. Unmentioned mouse buttons are assumed to have a 'released' state." },
+ { "name": "location", "$ref": "Point", "optional": true, "description": "For 'mouse' or 'touch' input sources, specifies a location in view coordinates to which the input source should transition. Transitioning to this state may interpolate intemediate input source states to better simulate real user movements and gestures." },
+ { "name": "duration", "type": "integer", "optional": true, "description": "The minimum number of milliseconds that must elapse while the relevant input source transitions to this state." }
+ ]
}
],
"commands": [
@@ -414,6 +455,24 @@
"async": true
},
{
+ "name": "performInteractionSequence",
+ "description": "Perform multiple simulated interactions over time using a list of input sources and a list of steps, where each step specifies a state for each input source at the time that step is performed.",
+ "parameters": [
+ { "name": "handle", "$ref": "BrowsingContextHandle", "description": "The browsing context to be interacted with." },
+ { "name": "inputSources", "type": "array", "items": { "$ref": "InputSource" }, "description": "All input sources that are used to perform this interaction sequence." },
+ { "name": "steps", "type": "array", "items": { "$ref": "InteractionStep" }, "description": "A list of steps that are executed in order." }
+ ],
+ "async": true
+ },
+ {
+ "name": "cancelInteractionSequence",
+ "description": "Cancel an active interaction sequence that is currently in progress.",
+ "parameters": [
+ { "name": "handle", "$ref": "BrowsingContextHandle", "description": "The browsing context to be interacted with." }
+ ],
+ "async": true
+ },
+ {
"name": "takeScreenshot",
"description": "Take a screenshot of the current page or given element in a browsing context.",
"parameters": [
Added: trunk/Source/WebKit/UIProcess/Automation/SimulatedInputDispatcher.cpp (0 => 230830)
--- trunk/Source/WebKit/UIProcess/Automation/SimulatedInputDispatcher.cpp (rev 0)
+++ trunk/Source/WebKit/UIProcess/Automation/SimulatedInputDispatcher.cpp 2018-04-20 01:45:21 UTC (rev 230830)
@@ -0,0 +1,272 @@
+/*
+ * Copyright (C) 2018 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "SimulatedInputDispatcher.h"
+
+#include "AutomationProtocolObjects.h"
+#include "WebAutomationSession.h"
+#include "WebAutomationSessionMacros.h"
+
+namespace WebKit {
+
+SimulatedInputKeyFrame::SimulatedInputKeyFrame(Vector<StateEntry>&& entries)
+ : states(WTFMove(entries))
+{
+}
+
+Seconds SimulatedInputKeyFrame::maximumDuration() const
+{
+ // The "compute the tick duration" algorithm (§17.4 Dispatching Actions).
+ Seconds result;
+ for (auto& entry : states)
+ result = std::max(result, entry.second.duration.value_or(Seconds(0)));
+
+ return result;
+}
+
+SimulatedInputKeyFrame SimulatedInputKeyFrame::keyFrameFromStateOfInputSources(HashSet<Ref<SimulatedInputSource>>& inputSources)
+{
+ // The client of this class is required to intern SimulatedInputSource instances if the last state
+ // from the previous command should be used as the inital state for the next command. This is the
+ // case for Perform Actions and Release Actions, but not Element Click or Element Send Keys.
+ Vector<SimulatedInputKeyFrame::StateEntry> entries;
+ entries.reserveCapacity(inputSources.size());
+
+ for (auto& inputSource : inputSources)
+ entries.uncheckedAppend(std::pair<SimulatedInputSource&, SimulatedInputSourceState> { inputSource.get(), inputSource->state });
+
+ return SimulatedInputKeyFrame(WTFMove(entries));
+}
+
+SimulatedInputKeyFrame SimulatedInputKeyFrame::keyFrameToResetInputSources(HashSet<Ref<SimulatedInputSource>>& inputSources)
+{
+ Vector<SimulatedInputKeyFrame::StateEntry> entries;
+ entries.reserveCapacity(inputSources.size());
+
+ for (auto& inputSource : inputSources)
+ entries.uncheckedAppend(std::pair<SimulatedInputSource&, SimulatedInputSourceState> { inputSource.get(), SimulatedInputSourceState::emptyState() });
+
+ return SimulatedInputKeyFrame(WTFMove(entries));
+}
+
+SimulatedInputDispatcher::SimulatedInputDispatcher(WebPageProxy& page, SimulatedInputDispatcher::Client& client)
+ : m_page(page)
+ , m_client(client)
+ , m_keyFrameTransitionDurationTimer(RunLoop::current(), this, &SimulatedInputDispatcher::keyFrameTransitionDurationTimerFired)
+{
+}
+
+SimulatedInputDispatcher::~SimulatedInputDispatcher()
+{
+ ASSERT(!m_runCompletionHandler);
+ ASSERT(!m_keyFrameTransitionDurationTimer.isActive());
+}
+
+bool SimulatedInputDispatcher::isActive() const
+{
+ return !!m_runCompletionHandler;
+}
+
+void SimulatedInputDispatcher::keyFrameTransitionDurationTimerFired()
+{
+ ASSERT(m_keyFrameTransitionCompletionHandler);
+
+ m_keyFrameTransitionDurationTimer.stop();
+
+ if (isKeyFrameTransitionComplete()) {
+ auto finish = std::exchange(m_keyFrameTransitionCompletionHandler, nullptr);
+ finish(std::nullopt);
+ }
+}
+
+bool SimulatedInputDispatcher::isKeyFrameTransitionComplete() const
+{
+ ASSERT(m_keyframeIndex < m_keyframes.size());
+
+ if (m_inputSourceStateIndex < m_keyframes[m_keyframeIndex].states.size())
+ return false;
+
+ if (m_keyFrameTransitionDurationTimer.isActive())
+ return false;
+
+ return true;
+}
+
+void SimulatedInputDispatcher::transitionToNextKeyFrame()
+{
+ ++m_keyframeIndex;
+ if (m_keyframeIndex == m_keyframes.size()) {
+ finishDispatching(std::nullopt);
+ return;
+ }
+
+ transitionBetweenKeyFrames(m_keyframes[m_keyframeIndex - 1], m_keyframes[m_keyframeIndex], [this, protectedThis = makeRef(*this)](std::optional<AutomationCommandError> error) {
+ if (error) {
+ finishDispatching(error);
+ return;
+ }
+
+ transitionToNextKeyFrame();
+ });
+}
+
+void SimulatedInputDispatcher::transitionToNextInputSourceState()
+{
+ if (isKeyFrameTransitionComplete()) {
+ auto finish = std::exchange(m_keyFrameTransitionCompletionHandler, nullptr);
+ finish(std::nullopt);
+ return;
+ }
+
+ // In this case, transitions are done but we need to wait for the tick timer.
+ if (m_inputSourceStateIndex == m_keyframes[m_keyframeIndex].states.size())
+ return;
+
+ auto& nextKeyFrame = m_keyframes[m_keyframeIndex];
+ auto& postStateEntry = nextKeyFrame.states[m_inputSourceStateIndex];
+ SimulatedInputSource& inputSource = postStateEntry.first;
+
+ transitionInputSourceToState(inputSource, postStateEntry.second, [this, protectedThis = makeRef(*this)](std::optional<AutomationCommandError> error) {
+ if (error) {
+ auto finish = std::exchange(m_keyFrameTransitionCompletionHandler, nullptr);
+ finish(error);
+ return;
+ }
+
+ // Perform state transitions in the order specified by the currentKeyFrame.
+ ++m_inputSourceStateIndex;
+
+ transitionToNextInputSourceState();
+ });
+}
+
+void SimulatedInputDispatcher::transitionBetweenKeyFrames(const SimulatedInputKeyFrame& a, const SimulatedInputKeyFrame& b, AutomationCompletionHandler&& completionHandler)
+{
+ m_inputSourceStateIndex = 0;
+
+ // The "dispatch tick actions" algorithm (§17.4 Dispatching Actions).
+ m_keyFrameTransitionCompletionHandler = WTFMove(completionHandler);
+ m_keyFrameTransitionDurationTimer.startOneShot(b.maximumDuration());
+
+ transitionToNextInputSourceState();
+}
+
+void SimulatedInputDispatcher::transitionInputSourceToState(SimulatedInputSource& inputSource, const SimulatedInputSourceState& newState, AutomationCompletionHandler&& completionHandler)
+{
+ // Make cases and conditionals more readable by aliasing pre/post states as 'a' and 'b'.
+ SimulatedInputSourceState a = inputSource.state;
+ SimulatedInputSourceState b = newState;
+
+ AutomationCompletionHandler eventDispatchFinished = [&inputSource, &newState, completionHandler = WTFMove(completionHandler)](std::optional<AutomationCommandError> error) {
+ if (error) {
+ completionHandler(error);
+ return;
+ }
+
+ inputSource.state = newState;
+ completionHandler(std::nullopt);
+ };
+
+ switch (inputSource.type) {
+ case SimulatedInputSource::Type::Null:
+ // The maximum duration is handled at the keyframe level by m_keyFrameTransitionDurationTimer.
+ eventDispatchFinished(std::nullopt);
+ break;
+ case SimulatedInputSource::Type::Mouse: {
+ // The "dispatch a pointer{Down,Up,Move} action" algorithms (§17.4 Dispatching Actions).
+ if (!a.pressedMouseButton && b.pressedMouseButton)
+ m_client.simulateMouseInteraction(m_page, MouseInteraction::Down, b.pressedMouseButton.value(), b.location.value(), WTFMove(eventDispatchFinished));
+ else if (a.pressedMouseButton && !b.pressedMouseButton)
+ m_client.simulateMouseInteraction(m_page, MouseInteraction::Up, a.pressedMouseButton.value(), b.location.value(), WTFMove(eventDispatchFinished));
+ else if (a.location != b.location) {
+ // FIXME: This does not interpolate mousemoves per the "perform a pointer move" algorithm (§17.4 Dispatching Actions).
+ m_client.simulateMouseInteraction(m_page, MouseInteraction::Move, b.pressedMouseButton.value_or(MouseButton::NoButton), b.location.value(), WTFMove(eventDispatchFinished));
+ } else
+ eventDispatchFinished(std::nullopt);
+ break;
+ }
+ case SimulatedInputSource::Type::Keyboard:
+ // The "dispatch a key{Down,Up} action" algorithms (§17.4 Dispatching Actions).
+ if ((!a.pressedCharKey && b.pressedCharKey) || (!a.pressedVirtualKey && b.pressedVirtualKey))
+ m_client.simulateKeyboardInteraction(m_page, KeyboardInteraction::KeyPress, b.pressedVirtualKey, b.pressedCharKey, WTFMove(eventDispatchFinished));
+ else if ((a.pressedCharKey && !b.pressedCharKey) || (a.pressedVirtualKey && !b.pressedVirtualKey))
+ m_client.simulateKeyboardInteraction(m_page, KeyboardInteraction::KeyRelease, a.pressedVirtualKey, a.pressedCharKey, WTFMove(eventDispatchFinished));
+ else
+ eventDispatchFinished(std::nullopt);
+ break;
+ case SimulatedInputSource::Type::Touch:
+ // Not supported yet.
+ ASSERT_NOT_REACHED();
+ eventDispatchFinished(AUTOMATION_COMMAND_ERROR_WITH_NAME(NotImplemented));
+ break;
+ }
+}
+
+void SimulatedInputDispatcher::run(Vector<SimulatedInputKeyFrame>&& keyFrames, HashSet<Ref<SimulatedInputSource>>& inputSources, AutomationCompletionHandler&& completionHandler)
+{
+ ASSERT(!isActive());
+ if (isActive()) {
+ completionHandler(AUTOMATION_COMMAND_ERROR_WITH_NAME(InternalError));
+ return;
+ }
+
+ m_runCompletionHandler = WTFMove(completionHandler);
+ for (const Ref<SimulatedInputSource>& inputSource : inputSources)
+ m_inputSources.add(inputSource.copyRef());
+
+ // The "dispatch actions" algorithm (§17.4 Dispatching Actions).
+
+ m_keyframes.reserveCapacity(keyFrames.size() + 1);
+ m_keyframes.append(SimulatedInputKeyFrame::keyFrameFromStateOfInputSources(m_inputSources));
+ m_keyframes.appendVector(WTFMove(keyFrames));
+
+ transitionToNextKeyFrame();
+}
+
+void SimulatedInputDispatcher::cancel()
+{
+ // If we were waiting for m_client to finish an interaction and the interaction had an error,
+ // then the rest of the async chain will have been torn down. If we are just waiting on a
+ // dispatch timer, then this will cancel the timer and clear
+
+ if (isActive())
+ finishDispatching(AUTOMATION_COMMAND_ERROR_WITH_NAME(InternalError));
+}
+
+void SimulatedInputDispatcher::finishDispatching(std::optional<AutomationCommandError> error)
+{
+ m_keyFrameTransitionDurationTimer.stop();
+
+ auto finish = std::exchange(m_runCompletionHandler, nullptr);
+ m_keyframes.clear();
+ m_inputSources.clear();
+ m_keyframeIndex = 0;
+ m_inputSourceStateIndex = 0;
+
+ finish(error);
+}
+
+} // namespace Webkit
Added: trunk/Source/WebKit/UIProcess/Automation/SimulatedInputDispatcher.h (0 => 230830)
--- trunk/Source/WebKit/UIProcess/Automation/SimulatedInputDispatcher.h (rev 0)
+++ trunk/Source/WebKit/UIProcess/Automation/SimulatedInputDispatcher.h 2018-04-20 01:45:21 UTC (rev 230830)
@@ -0,0 +1,160 @@
+/*
+ * Copyright (C) 2018 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#include "WebEvent.h"
+#include <wtf/CompletionHandler.h>
+#include <wtf/HashSet.h>
+#include <wtf/Optional.h>
+#include <wtf/RefCounted.h>
+#include <wtf/RunLoop.h>
+#include <wtf/Seconds.h>
+#include <wtf/Vector.h>
+#include <wtf/text/WTFString.h>
+
+namespace Inspector { namespace Protocol { namespace Automation {
+enum class ErrorMessage;
+enum class KeyboardInteractionType;
+enum class MouseInteraction;
+enum class VirtualKey;
+} } }
+
+namespace WebKit {
+
+class AutomationCommandError;
+using AutomationCompletionHandler = WTF::CompletionHandler<void(std::optional<AutomationCommandError>)>;
+
+class WebPageProxy;
+
+using KeyboardInteraction = Inspector::Protocol::Automation::KeyboardInteractionType;
+using VirtualKey = Inspector::Protocol::Automation::VirtualKey;
+using CharKey = char; // For WebDriver, this only needs to support ASCII characters on 102-key keyboard.
+using MouseButton = WebMouseEvent::Button;
+using MouseInteraction = Inspector::Protocol::Automation::MouseInteraction;
+
+struct SimulatedInputSourceState {
+ std::optional<CharKey> pressedCharKey;
+ std::optional<VirtualKey> pressedVirtualKey;
+ std::optional<MouseButton> pressedMouseButton;
+ std::optional<WebCore::IntPoint> location;
+ std::optional<Seconds> duration;
+
+ static SimulatedInputSourceState emptyState() { return SimulatedInputSourceState(); }
+};
+
+struct SimulatedInputSource : public RefCounted<SimulatedInputSource> {
+public:
+ enum class Type {
+ Null, // Used to induce a minimum duration.
+ Keyboard,
+ Mouse,
+ Touch,
+ };
+
+ Type type;
+
+ // The last state associated with this input source.
+ SimulatedInputSourceState state;
+
+ static Ref<SimulatedInputSource> create(Type type)
+ {
+ return adoptRef(*new SimulatedInputSource(type));
+ }
+
+private:
+ SimulatedInputSource(Type type)
+ : type(type)
+ , state(SimulatedInputSourceState::emptyState())
+ { }
+};
+
+struct SimulatedInputKeyFrame {
+public:
+ using StateEntry = std::pair<SimulatedInputSource&, SimulatedInputSourceState>;
+
+ explicit SimulatedInputKeyFrame(Vector<StateEntry>&&);
+ Seconds maximumDuration() const;
+
+ static SimulatedInputKeyFrame keyFrameFromStateOfInputSources(HashSet<Ref<SimulatedInputSource>>&);
+ static SimulatedInputKeyFrame keyFrameToResetInputSources(HashSet<Ref<SimulatedInputSource>>&);
+
+ Vector<StateEntry> states;
+};
+
+class SimulatedInputDispatcher : public RefCounted<SimulatedInputDispatcher> {
+ WTF_MAKE_NONCOPYABLE(SimulatedInputDispatcher);
+public:
+ class Client {
+ public:
+ virtual ~Client() { }
+ virtual void simulateMouseInteraction(WebPageProxy&, MouseInteraction, WebMouseEvent::Button, const WebCore::IntPoint& locationInView, AutomationCompletionHandler&&) = 0;
+ virtual void simulateKeyboardInteraction(WebPageProxy&, KeyboardInteraction, std::optional<VirtualKey>, std::optional<CharKey>, AutomationCompletionHandler&&) = 0;
+ };
+
+ static Ref<SimulatedInputDispatcher> create(WebPageProxy& page, SimulatedInputDispatcher::Client& client)
+ {
+ return adoptRef(*new SimulatedInputDispatcher(page, client));
+ }
+
+ ~SimulatedInputDispatcher();
+
+ void run(Vector<SimulatedInputKeyFrame>&& keyFrames, HashSet<Ref<SimulatedInputSource>>& inputSources, AutomationCompletionHandler&&);
+ void cancel();
+
+ bool isActive() const;
+
+private:
+ SimulatedInputDispatcher(WebPageProxy&, SimulatedInputDispatcher::Client&);
+
+ void transitionToNextKeyFrame();
+ void transitionBetweenKeyFrames(const SimulatedInputKeyFrame&, const SimulatedInputKeyFrame&, AutomationCompletionHandler&&);
+
+ void transitionToNextInputSourceState();
+ void transitionInputSourceToState(SimulatedInputSource&, const SimulatedInputSourceState& newState, AutomationCompletionHandler&&);
+ void finishDispatching(std::optional<AutomationCommandError>);
+
+ void keyFrameTransitionDurationTimerFired();
+ bool isKeyFrameTransitionComplete() const;
+
+ WebPageProxy& m_page;
+ SimulatedInputDispatcher::Client& m_client;
+
+ AutomationCompletionHandler m_runCompletionHandler;
+ AutomationCompletionHandler m_keyFrameTransitionCompletionHandler;
+ RunLoop::Timer<SimulatedInputDispatcher> m_keyFrameTransitionDurationTimer;
+
+ Vector<SimulatedInputKeyFrame> m_keyframes;
+ HashSet<Ref<SimulatedInputSource>> m_inputSources;
+
+ // The position within m_keyframes.
+ unsigned m_keyframeIndex { 0 };
+
+ // The position within the input source state vector at m_keyframes[m_keyframeIndex].
+ // Events that reflect input source state transitions are dispatched serially based on this order.
+ unsigned m_inputSourceStateIndex { 0 };
+};
+
+} // namespace WebKit
Modified: trunk/Source/WebKit/UIProcess/Automation/WebAutomationSession.cpp (230829 => 230830)
--- trunk/Source/WebKit/UIProcess/Automation/WebAutomationSession.cpp 2018-04-20 01:00:40 UTC (rev 230829)
+++ trunk/Source/WebKit/UIProcess/Automation/WebAutomationSession.cpp 2018-04-20 01:45:21 UTC (rev 230830)
@@ -76,6 +76,10 @@
, m_domainNotifier(std::make_unique<AutomationFrontendDispatcher>(m_frontendRouter))
, m_loadTimer(RunLoop::main(), this, &WebAutomationSession::loadTimerFired)
{
+ // Set up canonical input sources to be used for 'performInteractionSequence' and 'cancelInteractionSequence'.
+ m_inputSources.add(SimulatedInputSource::create(SimulatedInputSource::Type::Mouse));
+ m_inputSources.add(SimulatedInputSource::create(SimulatedInputSource::Type::Keyboard));
+ m_inputSources.add(SimulatedInputSource::create(SimulatedInputSource::Type::Null));
}
WebAutomationSession::~WebAutomationSession()
@@ -723,22 +727,14 @@
void WebAutomationSession::mouseEventsFlushedForPage(const WebPageProxy& page)
{
- if (auto callback = m_pendingMouseEventsFlushedCallbacksPerPage.take(page.pageID())) {
- if (m_pendingMouseEventsFlushedCallbacksPerPage.isEmpty())
- m_simulatingUserInteraction = false;
-
+ if (auto callback = m_pendingMouseEventsFlushedCallbacksPerPage.take(page.pageID()))
callback(std::nullopt);
- }
}
void WebAutomationSession::keyboardEventsFlushedForPage(const WebPageProxy& page)
{
- if (auto callback = m_pendingKeyboardEventsFlushedCallbacksPerPage.take(page.pageID())) {
- if (m_pendingKeyboardEventsFlushedCallbacksPerPage.isEmpty())
- m_simulatingUserInteraction = false;
-
+ if (auto callback = m_pendingKeyboardEventsFlushedCallbacksPerPage.take(page.pageID()))
callback(std::nullopt);
- }
}
void WebAutomationSession::willClosePage(const WebPageProxy& page)
@@ -745,6 +741,18 @@
{
String handle = handleForWebPageProxy(page);
m_domainNotifier->browsingContextCleared(handle);
+
+ // Cancel pending interactions on this page. By providing an error, this will cause subsequent
+ // actions to be aborted and the SimulatedInputDispatcher::run() call will unwind and fail.
+ if (auto callback = m_pendingMouseEventsFlushedCallbacksPerPage.take(page.pageID()))
+ callback(AUTOMATION_COMMAND_ERROR_WITH_NAME(WindowNotFound));
+ if (auto callback = m_pendingMouseEventsFlushedCallbacksPerPage.take(page.pageID()))
+ callback(AUTOMATION_COMMAND_ERROR_WITH_NAME(WindowNotFound));
+
+ // Then tell the input dispatcher to cancel so timers are stopped, and let it go out of scope.
+ std::optional<Ref<SimulatedInputDispatcher>> inputDispatcher = m_inputDispatchersByPage.take(page.pageID());
+ if (inputDispatcher.has_value())
+ inputDispatcher.value()->cancel();
}
static bool fileCanBeAcceptedForUpload(const String& filename, const HashSet<String>& allowedMIMETypes, const HashSet<String>& allowedFileExtensions)
@@ -1366,6 +1374,85 @@
return m_permissionForGetUserMedia;
}
+bool WebAutomationSession::isSimulatingUserInteraction() const
+{
+ if (!m_pendingMouseEventsFlushedCallbacksPerPage.isEmpty())
+ return false;
+ if (!m_pendingKeyboardEventsFlushedCallbacksPerPage.isEmpty())
+ return false;
+
+ return true;
+}
+
+SimulatedInputDispatcher& WebAutomationSession::inputDispatcherForPage(WebPageProxy& page)
+{
+ return m_inputDispatchersByPage.ensure(page.pageID(), [&] {
+ return SimulatedInputDispatcher::create(page, *this);
+ }).iterator->value;
+}
+
+SimulatedInputSource* WebAutomationSession::inputSourceForType(SimulatedInputSource::Type type) const
+{
+ // FIXME: this should use something like Vector's findMatching().
+ for (auto& inputSource : m_inputSources) {
+ if (inputSource->type == type)
+ return &inputSource.get();
+ }
+
+ return nullptr;
+}
+
+// SimulatedInputDispatcher::Client API
+void WebAutomationSession::simulateMouseInteraction(WebPageProxy& page, MouseInteraction interaction, WebMouseEvent::Button mouseButton, const WebCore::IntPoint& locationInViewport, CompletionHandler<void(std::optional<AutomationCommandError>)>&& completionHandler)
+{
+ WebCore::IntPoint locationInView = WebCore::IntPoint(locationInViewport.x(), locationInViewport.y() + page.topContentInset());
+ page.getWindowFrameWithCallback([this, protectedThis = makeRef(*this), completionHandler = WTFMove(completionHandler), page = makeRef(page), interaction, mouseButton, locationInView](WebCore::FloatRect windowFrame) mutable {
+ auto clippedX = std::min(std::max(0.0f, (float)locationInView.x()), windowFrame.size().width());
+ auto clippedY = std::min(std::max(0.0f, (float)locationInView.y()), windowFrame.size().height());
+ if (clippedX != locationInView.x() || clippedY != locationInView.y()) {
+ completionHandler(AUTOMATION_COMMAND_ERROR_WITH_NAME(TargetOutOfBounds));
+ return;
+ }
+
+ // Bridge the flushed callback to our command's completion handler.
+ auto mouseEventsFlushedCallback = [completionHandler = WTFMove(completionHandler)](std::optional<AutomationCommandError> error) {
+ completionHandler(error);
+ };
+
+ auto& callbackInMap = m_pendingMouseEventsFlushedCallbacksPerPage.add(page->pageID(), nullptr).iterator->value;
+ if (callbackInMap)
+ callbackInMap(AUTOMATION_COMMAND_ERROR_WITH_NAME(Timeout));
+ callbackInMap = WTFMove(mouseEventsFlushedCallback);
+
+ platformSimulateMouseInteraction(page, interaction, mouseButton, locationInView, (WebEvent::Modifiers)m_currentModifiers);
+
+ // If the event does not hit test anything in the window, then it may not have been delivered.
+ if (callbackInMap && !page->isProcessingMouseEvents()) {
+ auto callbackToCancel = m_pendingMouseEventsFlushedCallbacksPerPage.take(page->pageID());
+ callbackToCancel(std::nullopt);
+ }
+
+ // Otherwise, wait for mouseEventsFlushedCallback to run when all events are handled.
+ });
+}
+
+void WebAutomationSession::simulateKeyboardInteraction(WebPageProxy& page, KeyboardInteraction interaction, std::optional<VirtualKey> virtualKey, std::optional<CharKey> charKey, CompletionHandler<void(std::optional<AutomationCommandError>)>&& completionHandler)
+{
+ // Bridge the flushed callback to our command's completion handler.
+ auto keyboardEventsFlushedCallback = [completionHandler = WTFMove(completionHandler)](std::optional<AutomationCommandError> error) {
+ completionHandler(error);
+ };
+
+ auto& callbackInMap = m_pendingKeyboardEventsFlushedCallbacksPerPage.add(page.pageID(), nullptr).iterator->value;
+ if (callbackInMap)
+ callbackInMap(AUTOMATION_COMMAND_ERROR_WITH_NAME(Timeout));
+ callbackInMap = WTFMove(keyboardEventsFlushedCallback);
+
+ platformSimulateKeyboardInteraction(page, interaction, virtualKey, charKey);
+
+ // Wait for keyboardEventsFlushedCallback to run when all events are handled.
+}
+
#if USE(APPKIT) || PLATFORM(GTK)
static WebEvent::Modifiers protocolModifierToWebEventModifier(Inspector::Protocol::Automation::KeyModifier modifier)
{
@@ -1384,6 +1471,22 @@
RELEASE_ASSERT_NOT_REACHED();
}
+
+static WebMouseEvent::Button protocolMouseButtonToWebMouseEventButton(Inspector::Protocol::Automation::MouseButton button)
+{
+ switch (button) {
+ case Inspector::Protocol::Automation::MouseButton::None:
+ return WebMouseEvent::NoButton;
+ case Inspector::Protocol::Automation::MouseButton::Left:
+ return WebMouseEvent::LeftButton;
+ case Inspector::Protocol::Automation::MouseButton::Middle:
+ return WebMouseEvent::MiddleButton;
+ case Inspector::Protocol::Automation::MouseButton::Right:
+ return WebMouseEvent::RightButton;
+ }
+
+ RELEASE_ASSERT_NOT_REACHED();
+}
#endif // USE(APPKIT) || PLATFORM(GTK)
void WebAutomationSession::performMouseInteraction(const String& handle, const JSON::Object& requestedPositionObject, const String& mouseButtonString, const String& mouseInteractionString, const JSON::Array& keyModifierStrings, Ref<PerformMouseInteractionCallback>&& callback)
@@ -1415,13 +1518,13 @@
WebEvent::Modifiers enumValue = protocolModifierToWebEventModifier(parsedModifier.value());
keyModifiers = (WebEvent::Modifiers)(enumValue | keyModifiers);
}
-
+
page->getWindowFrameWithCallback([this, protectedThis = makeRef(*this), callback = WTFMove(callback), page = makeRef(*page), x, y, mouseInteractionString, mouseButtonString, keyModifiers](WebCore::FloatRect windowFrame) mutable {
x = std::min(std::max(0.0f, x), windowFrame.size().width());
y = std::min(std::max(0.0f, y + page->topContentInset()), windowFrame.size().height());
- WebCore::IntPoint viewPosition = WebCore::IntPoint(static_cast<int>(x), static_cast<int>(y));
+ WebCore::IntPoint positionInView = WebCore::IntPoint(static_cast<int>(x), static_cast<int>(y));
auto parsedInteraction = Inspector::Protocol::AutomationHelpers::parseEnumValueFromString<Inspector::Protocol::Automation::MouseInteraction>(mouseInteractionString);
if (!parsedInteraction)
@@ -1447,11 +1550,8 @@
callbackInMap(AUTOMATION_COMMAND_ERROR_WITH_NAME(Timeout));
callbackInMap = WTFMove(mouseEventsFlushedCallback);
- // This is cleared when all mouse events are flushed.
- m_simulatingUserInteraction = true;
+ platformSimulateMouseInteraction(page, parsedInteraction.value(), protocolMouseButtonToWebMouseEventButton(parsedButton.value()), positionInView, keyModifiers);
- platformSimulateMouseInteraction(page, viewPosition, parsedInteraction.value(), parsedButton.value(), keyModifiers);
-
// If the event location was previously clipped and does not hit test anything in the window, then it will not be processed.
// For compatibility with pre-W3C driver implementations, don't make this a hard error; just do nothing silently.
// In W3C-only code paths, we can reject any pointer actions whose coordinates are outside the viewport rect.
@@ -1499,7 +1599,7 @@
ASYNC_FAIL_WITH_PREDEFINED_ERROR_AND_DETAILS(InvalidParameter, "An interaction in the 'interactions' parameter has an invalid 'key' value.");
actionsToPerform.uncheckedAppend([this, page, interactionType, virtualKey] {
- platformSimulateKeyStroke(*page, interactionType.value(), virtualKey.value());
+ platformSimulateKeyboardInteraction(*page, interactionType.value(), virtualKey, std::nullopt);
});
}
@@ -1540,14 +1640,181 @@
callbackInMap(AUTOMATION_COMMAND_ERROR_WITH_NAME(Timeout));
callbackInMap = WTFMove(keyboardEventsFlushedCallback);
- // This is cleared when all keyboard events are flushed.
- m_simulatingUserInteraction = true;
-
for (auto& action : actionsToPerform)
action();
#endif // PLATFORM(COCOA) || PLATFORM(GTK)
}
+#if USE(APPKIT) || PLATFORM(GTK)
+static SimulatedInputSource::Type simulatedInputSourceTypeFromProtocolSourceType(Inspector::Protocol::Automation::InputSourceType protocolType)
+{
+ switch (protocolType) {
+ case Inspector::Protocol::Automation::InputSourceType::Null:
+ return SimulatedInputSource::Type::Null;
+ case Inspector::Protocol::Automation::InputSourceType::Keyboard:
+ return SimulatedInputSource::Type::Keyboard;
+ case Inspector::Protocol::Automation::InputSourceType::Mouse:
+ return SimulatedInputSource::Type::Mouse;
+ case Inspector::Protocol::Automation::InputSourceType::Touch:
+ return SimulatedInputSource::Type::Touch;
+ }
+
+ RELEASE_ASSERT_NOT_REACHED();
+}
+#endif // USE(APPKIT) || PLATFORM(GTK)
+
+void WebAutomationSession::performInteractionSequence(const String& handle, const JSON::Array& inputSources, const JSON::Array& steps, Ref<WebAutomationSession::PerformInteractionSequenceCallback>&& callback)
+{
+ // This command implements WebKit support for §17.5 Perform Actions.
+
+#if !USE(APPKIT) && !PLATFORM(GTK)
+ ASYNC_FAIL_WITH_PREDEFINED_ERROR(NotImplemented);
+#else
+ WebPageProxy* page = webPageProxyForHandle(handle);
+ if (!page)
+ ASYNC_FAIL_WITH_PREDEFINED_ERROR(WindowNotFound);
+
+ HashMap<String, Ref<SimulatedInputSource>> sourceIdToInputSourceMap;
+ HashMap<SimulatedInputSource::Type, String, WTF::IntHash<SimulatedInputSource::Type>, WTF::StrongEnumHashTraits<SimulatedInputSource::Type>> typeToSourceIdMap;
+
+ // Parse and validate Automation protocol arguments. By this point, the driver has
+ // already performed the steps in §17.3 Processing Actions Requests.
+ if (!inputSources.length())
+ ASYNC_FAIL_WITH_PREDEFINED_ERROR_AND_DETAILS(InvalidParameter, "The parameter 'inputSources' was not found or empty.");
+
+ for (auto inputSource : inputSources) {
+ RefPtr<JSON::Object> inputSourceObject;
+ if (!inputSource->asObject(inputSourceObject))
+ ASYNC_FAIL_WITH_PREDEFINED_ERROR_AND_DETAILS(InvalidParameter, "An input source in the 'inputSources' parameter was invalid.");
+
+ String sourceId;
+ if (!inputSourceObject->getString(ASCIILiteral("sourceId"), sourceId))
+ ASYNC_FAIL_WITH_PREDEFINED_ERROR_AND_DETAILS(InvalidParameter, "An input source in the 'inputSources' parameter is missing a 'sourceId'.");
+
+ String sourceType;
+ if (!inputSourceObject->getString(ASCIILiteral("sourceType"), sourceType))
+ ASYNC_FAIL_WITH_PREDEFINED_ERROR_AND_DETAILS(InvalidParameter, "An input source in the 'inputSources' parameter is missing a 'sourceType'.");
+
+ auto parsedInputSourceType = Inspector::Protocol::AutomationHelpers::parseEnumValueFromString<Inspector::Protocol::Automation::InputSourceType>(sourceType);
+ if (!parsedInputSourceType)
+ ASYNC_FAIL_WITH_PREDEFINED_ERROR_AND_DETAILS(InvalidParameter, "An input source in the 'inputSources' parameter has an invalid 'sourceType'.");
+
+ SimulatedInputSource::Type inputSourceType = simulatedInputSourceTypeFromProtocolSourceType(*parsedInputSourceType);
+ if (inputSourceType == SimulatedInputSource::Type::Touch)
+ ASYNC_FAIL_WITH_PREDEFINED_ERROR_AND_DETAILS(NotImplemented, "Touch input sources are not yet supported.");
+
+ if (typeToSourceIdMap.contains(inputSourceType))
+ ASYNC_FAIL_WITH_PREDEFINED_ERROR_AND_DETAILS(InvalidParameter, "Two input sources with the same type were specified.");
+ if (sourceIdToInputSourceMap.contains(sourceId))
+ ASYNC_FAIL_WITH_PREDEFINED_ERROR_AND_DETAILS(InvalidParameter, "Two input sources with the same sourceId were specified.");
+
+ typeToSourceIdMap.add(inputSourceType, sourceId);
+ sourceIdToInputSourceMap.add(sourceId, *inputSourceForType(inputSourceType));
+ }
+
+ Vector<SimulatedInputKeyFrame> keyFrames;
+
+ if (!steps.length())
+ ASYNC_FAIL_WITH_PREDEFINED_ERROR_AND_DETAILS(InvalidParameter, "The parameter 'steps' was not found or empty.");
+
+ for (auto step : steps) {
+ RefPtr<JSON::Object> stepObject;
+ if (!step->asObject(stepObject))
+ ASYNC_FAIL_WITH_PREDEFINED_ERROR_AND_DETAILS(InvalidParameter, "A step in the 'steps' parameter was not an object.");
+
+ RefPtr<JSON::Array> stepStates;
+ if (!stepObject->getArray(ASCIILiteral("states"), stepStates))
+ ASYNC_FAIL_WITH_PREDEFINED_ERROR_AND_DETAILS(InvalidParameter, "A step is missing the 'states' property.");
+
+ Vector<SimulatedInputKeyFrame::StateEntry> entries;
+ entries.reserveCapacity(stepStates->length());
+
+ for (auto state : *stepStates) {
+ RefPtr<JSON::Object> stateObject;
+ if (!state->asObject(stateObject))
+ ASYNC_FAIL_WITH_PREDEFINED_ERROR_AND_DETAILS(InvalidParameter, "Encountered a non-object step state.");
+
+ String sourceId;
+ if (!stateObject->getString(ASCIILiteral("sourceId"), sourceId))
+ ASYNC_FAIL_WITH_PREDEFINED_ERROR_AND_DETAILS(InvalidParameter, "Step state lacks required 'sourceId' property.");
+
+ if (!sourceIdToInputSourceMap.contains(sourceId))
+ ASYNC_FAIL_WITH_PREDEFINED_ERROR_AND_DETAILS(InvalidParameter, "Unknown 'sourceId' specified.");
+
+ SimulatedInputSource& inputSource = *sourceIdToInputSourceMap.get(sourceId);
+ SimulatedInputSourceState sourceState { };
+
+ String pressedCharKeyString;
+ if (stateObject->getString(ASCIILiteral("pressedCharKey"), pressedCharKeyString))
+ sourceState.pressedCharKey = pressedCharKeyString.characterAt(0);
+
+ String pressedVirtualKeyString;
+ if (stateObject->getString(ASCIILiteral("pressedVirtualKey"), pressedVirtualKeyString))
+ sourceState.pressedVirtualKey = Inspector::Protocol::AutomationHelpers::parseEnumValueFromString<Inspector::Protocol::Automation::VirtualKey>(pressedVirtualKeyString);
+
+ String pressedButtonString;
+ if (stateObject->getString(ASCIILiteral("pressedButton"), pressedButtonString)) {
+ auto protocolButton = Inspector::Protocol::AutomationHelpers::parseEnumValueFromString<Inspector::Protocol::Automation::MouseButton>(pressedButtonString);
+ sourceState.pressedMouseButton = protocolMouseButtonToWebMouseEventButton(protocolButton.value_or(Inspector::Protocol::Automation::MouseButton::None));
+ }
+
+ RefPtr<JSON::Object> locationObject;
+ if (stateObject->getObject(ASCIILiteral("location"), locationObject)) {
+ int x, y;
+ if (locationObject->getInteger(ASCIILiteral("x"), x) && locationObject->getInteger(ASCIILiteral("y"), y))
+ sourceState.location = WebCore::IntPoint(x, y);
+ }
+
+ int parsedDuration;
+ if (stateObject->getInteger(ASCIILiteral("duration"), parsedDuration))
+ sourceState.duration = Seconds::fromMilliseconds(parsedDuration);
+
+ entries.uncheckedAppend(std::pair<SimulatedInputSource&, SimulatedInputSourceState> { inputSource, sourceState });
+ }
+
+ keyFrames.append(SimulatedInputKeyFrame(WTFMove(entries)));
+ }
+
+ SimulatedInputDispatcher& inputDispatcher = inputDispatcherForPage(*page);
+ if (inputDispatcher.isActive()) {
+ ASSERT_NOT_REACHED();
+ ASYNC_FAIL_WITH_PREDEFINED_ERROR_AND_DETAILS(InternalError, "A previous interaction is still underway.");
+ }
+
+ // Delegate the rest of §17.4 Dispatching Actions to the dispatcher.
+ inputDispatcher.run(WTFMove(keyFrames), m_inputSources, [protectedThis = makeRef(*this), callback = WTFMove(callback)](std::optional<AutomationCommandError> error) {
+ if (error)
+ callback->sendFailure(error.value().toProtocolString());
+ else
+ callback->sendSuccess();
+ });
+#endif // PLATFORM(COCOA) || PLATFORM(GTK)
+}
+
+void WebAutomationSession::cancelInteractionSequence(const String& handle, Ref<CancelInteractionSequenceCallback>&& callback)
+{
+ // This command implements WebKit support for §17.6 Release Actions.
+
+#if !USE(APPKIT) && !PLATFORM(GTK)
+ ASYNC_FAIL_WITH_PREDEFINED_ERROR(NotImplemented);
+#else
+ WebPageProxy* page = webPageProxyForHandle(handle);
+ if (!page)
+ ASYNC_FAIL_WITH_PREDEFINED_ERROR(WindowNotFound);
+
+ Vector<SimulatedInputKeyFrame> keyFrames({ SimulatedInputKeyFrame::keyFrameToResetInputSources(m_inputSources) });
+ SimulatedInputDispatcher& inputDispatcher = inputDispatcherForPage(*page);
+ inputDispatcher.cancel();
+
+ inputDispatcher.run(WTFMove(keyFrames), m_inputSources, [protectedThis = makeRef(*this), callback = WTFMove(callback)](std::optional<AutomationCommandError> error) {
+ if (error)
+ callback->sendFailure(error.value().toProtocolString());
+ else
+ callback->sendSuccess();
+ });
+#endif // PLATFORM(COCOA) || PLATFORM(GTK)
+}
+
void WebAutomationSession::takeScreenshot(const String& handle, const String* optionalFrameHandle, const String* optionalNodeHandle, const bool* optionalScrollIntoViewIfNeeded, const bool* optionalClipToViewport, Ref<TakeScreenshotCallback>&& callback)
{
WebPageProxy* page = webPageProxyForHandle(handle);
@@ -1589,17 +1856,15 @@
// Platform-dependent Implementation Stubs.
#if !PLATFORM(MAC) && !PLATFORM(GTK)
-void WebAutomationSession::platformSimulateMouseInteraction(WebKit::WebPageProxy&, const WebCore::IntPoint&, Inspector::Protocol::Automation::MouseInteraction, Inspector::Protocol::Automation::MouseButton, WebEvent::Modifiers)
+void WebAutomationSession::platformSimulateMouseInteraction(WebPageProxy&, MouseInteraction, WebMouseEvent::Button, const WebCore::IntPoint&, WebEvent::Modifiers)
{
}
#endif // !PLATFORM(MAC) && !PLATFORM(GTK)
#if !PLATFORM(COCOA) && !PLATFORM(GTK)
-void WebAutomationSession::platformSimulateKeyStroke(WebPageProxy&, Inspector::Protocol::Automation::KeyboardInteractionType, Inspector::Protocol::Automation::VirtualKey)
-{
-}
-void WebAutomationSession::platformSimulateKeySequence(WebPageProxy&, const String&)
+
+void WebAutomationSession::platformSimulateKeyboardInteraction(WebPageProxy&, KeyboardInteraction, std::optional<VirtualKey>, std::optional<CharKey>)
{
}
#endif // !PLATFORM(COCOA) && !PLATFORM(GTK)
Modified: trunk/Source/WebKit/UIProcess/Automation/WebAutomationSession.h (230829 => 230830)
--- trunk/Source/WebKit/UIProcess/Automation/WebAutomationSession.h 2018-04-20 01:00:40 UTC (rev 230829)
+++ trunk/Source/WebKit/UIProcess/Automation/WebAutomationSession.h 2018-04-20 01:45:21 UTC (rev 230830)
@@ -30,6 +30,7 @@
#include "AutomationFrontendDispatchers.h"
#include "Connection.h"
#include "ShareableBitmap.h"
+#include "SimulatedInputDispatcher.h"
#include "WebEvent.h"
#include <wtf/CompletionHandler.h>
#include <wtf/Forward.h>
@@ -76,7 +77,7 @@
class WebPageProxy;
class WebProcessPool;
-struct AutomationCommandError {
+class AutomationCommandError {
public:
Inspector::Protocol::Automation::ErrorMessage type;
std::optional<String> message { std::nullopt };
@@ -91,11 +92,14 @@
String toProtocolString();
};
+using AutomationCompletionHandler = WTF::CompletionHandler<void(std::optional<AutomationCommandError>)>;
+
class WebAutomationSession final : public API::ObjectImpl<API::Object::Type::AutomationSession>, public IPC::MessageReceiver
#if ENABLE(REMOTE_INSPECTOR)
, public Inspector::RemoteAutomationTarget
#endif
, public Inspector::AutomationBackendDispatcherHandler
+ , public SimulatedInputDispatcher::Client
{
public:
WebAutomationSession();
@@ -131,6 +135,10 @@
#endif
void terminate();
+ // SimulatedInputDispatcher::Client API
+ void simulateMouseInteraction(WebPageProxy&, MouseInteraction, WebMouseEvent::Button, const WebCore::IntPoint& locationInView, AutomationCompletionHandler&&) final;
+ void simulateKeyboardInteraction(WebPageProxy&, KeyboardInteraction, std::optional<VirtualKey>, std::optional<CharKey>, AutomationCompletionHandler&&) final;
+
// Inspector::AutomationBackendDispatcherHandler API
// NOTE: the set of declarations included in this interface depend on the "platform" property in Automation.json
// and the --platform argument passed to the protocol bindings generator.
@@ -151,6 +159,8 @@
void evaluateJavaScriptFunction(const String& browsingContextHandle, const String* optionalFrameHandle, const String& function, const JSON::Array& arguments, const bool* optionalExpectsImplicitCallbackArgument, const int* optionalCallbackTimeout, Ref<Inspector::AutomationBackendDispatcherHandler::EvaluateJavaScriptFunctionCallback>&&) override;
void performMouseInteraction(const String& handle, const JSON::Object& requestedPosition, const String& mouseButton, const String& mouseInteraction, const JSON::Array& keyModifiers, Ref<PerformMouseInteractionCallback>&&) final;
void performKeyboardInteractions(const String& handle, const JSON::Array& interactions, Ref<PerformKeyboardInteractionsCallback>&&) override;
+ void performInteractionSequence(const String& handle, const JSON::Array& sources, const JSON::Array& steps, Ref<PerformInteractionSequenceCallback>&&) override;
+ void cancelInteractionSequence(const String& handle, Ref<CancelInteractionSequenceCallback>&&) override;
void takeScreenshot(const String& handle, const String* optionalFrameHandle, const String* optionalNodeHandle, const bool* optionalScrollIntoViewIfNeeded, const bool* optionalClipToViewport, Ref<TakeScreenshotCallback>&&) override;
void resolveChildFrameHandle(const String& browsingContextHandle, const String* optionalFrameHandle, const int* optionalOrdinal, const String* optionalName, const String* optionalNodeHandle, Ref<ResolveChildFrameHandleCallback>&&) override;
void resolveParentFrameHandle(const String& browsingContextHandle, const String& frameHandle, Ref<ResolveParentFrameHandleCallback>&&) override;
@@ -175,7 +185,10 @@
#endif
// Event Simulation Support.
- bool isSimulatingUserInteraction() const { return m_simulatingUserInteraction; }
+ bool isSimulatingUserInteraction() const;
+ SimulatedInputDispatcher& inputDispatcherForPage(WebPageProxy&);
+ SimulatedInputSource* inputSourceForType(SimulatedInputSource::Type) const;
+
#if PLATFORM(MAC)
bool wasEventSynthesizedForAutomation(NSEvent *);
void markEventAsSynthesizedForAutomation(NSEvent *);
@@ -215,9 +228,9 @@
void didDeleteCookie(uint64_t callbackID, const String& errorType);
// Platform-dependent implementations.
- void platformSimulateMouseInteraction(WebPageProxy&, const WebCore::IntPoint& viewPosition, Inspector::Protocol::Automation::MouseInteraction, Inspector::Protocol::Automation::MouseButton, WebEvent::Modifiers);
- // Simulates a single virtual key being pressed, such as Control, F-keys, Numpad keys, etc. as allowed by the protocol.
- void platformSimulateKeyStroke(WebPageProxy&, Inspector::Protocol::Automation::KeyboardInteractionType, Inspector::Protocol::Automation::VirtualKey);
+ void platformSimulateMouseInteraction(WebPageProxy&, MouseInteraction, WebMouseEvent::Button, const WebCore::IntPoint& locationInView, WebEvent::Modifiers keyModifiers);
+ // Simulates a single virtual or char key being pressed/released, such as 'a', Control, F-keys, Numpad keys, etc. as allowed by the protocol.
+ void platformSimulateKeyboardInteraction(WebPageProxy&, KeyboardInteraction, std::optional<VirtualKey>, std::optional<CharKey>);
// Simulates key presses to produce the codepoints in a string. One or more code points are delivered atomically at grapheme cluster boundaries.
void platformSimulateKeySequence(WebPageProxy&, const String&);
// Get base64 encoded PNG data from a bitmap.
@@ -289,13 +302,15 @@
bool m_permissionForGetUserMedia { true };
- // True if a synthesized key event is still being processed.
- bool m_simulatingUserInteraction { false };
-
// Keep track of currently active modifiers across multiple keystrokes.
// Most platforms do not track current modifiers from synthesized events.
unsigned m_currentModifiers { 0 };
+ // SimulatedInputDispatcher APIs take a set of input sources. We also intern these
+ // so that previous input source state is used as initial state for later commands.
+ HashSet<Ref<SimulatedInputSource>> m_inputSources;
+ HashMap<uint64_t, Ref<SimulatedInputDispatcher>> m_inputDispatchersByPage;
+
#if ENABLE(REMOTE_INSPECTOR)
Inspector::FrontendChannel* m_remoteChannel { nullptr };
#endif
Modified: trunk/Source/WebKit/UIProcess/Automation/gtk/WebAutomationSessionGtk.cpp (230829 => 230830)
--- trunk/Source/WebKit/UIProcess/Automation/gtk/WebAutomationSessionGtk.cpp 2018-04-20 01:00:40 UTC (rev 230829)
+++ trunk/Source/WebKit/UIProcess/Automation/gtk/WebAutomationSessionGtk.cpp 2018-04-20 01:45:21 UTC (rev 230830)
@@ -48,15 +48,15 @@
return state;
}
-static unsigned mouseButtonToGdkButton(Inspector::Protocol::Automation::MouseButton button)
+static unsigned mouseButtonToGdkButton(WebMouseEvent::Button button)
{
switch (button) {
- case Inspector::Protocol::Automation::MouseButton::None:
- case Inspector::Protocol::Automation::MouseButton::Left:
+ case WebMouseEvent::NoButton:
+ case WebMouseEvent::LeftButton:
return GDK_BUTTON_PRIMARY;
- case Inspector::Protocol::Automation::MouseButton::Middle:
+ case WebMouseEvent::MiddleButton:
return GDK_BUTTON_MIDDLE;
- case Inspector::Protocol::Automation::MouseButton::Right:
+ case WebMouseEvent::RightButton:
return GDK_BUTTON_SECONDARY;
}
return GDK_BUTTON_PRIMARY;
@@ -101,30 +101,30 @@
gtk_main_do_event(event.get());
}
-void WebAutomationSession::platformSimulateMouseInteraction(WebPageProxy& page, const WebCore::IntPoint& viewPosition, Inspector::Protocol::Automation::MouseInteraction interaction, Inspector::Protocol::Automation::MouseButton button, WebEvent::Modifiers keyModifiers)
+void WebAutomationSession::platformSimulateMouseInteraction(WebPageProxy& page, MouseInteraction interaction, WebMouseEvent::Button button, const WebCore::IntPoint& locationInView, WebEvent::Modifiers keyModifiers)
{
unsigned gdkButton = mouseButtonToGdkButton(button);
unsigned state = modifiersToEventState(keyModifiers);
switch (interaction) {
- case Inspector::Protocol::Automation::MouseInteraction::Move:
- doMotionEvent(page.viewWidget(), viewPosition, state);
+ case MouseInteraction::Move:
+ doMotionEvent(page.viewWidget(), locationInView, state);
break;
- case Inspector::Protocol::Automation::MouseInteraction::Down:
- doMouseEvent(GDK_BUTTON_PRESS, page.viewWidget(), viewPosition, gdkButton, state);
+ case MouseInteraction::Down:
+ doMouseEvent(GDK_BUTTON_PRESS, page.viewWidget(), locationInView, gdkButton, state);
break;
- case Inspector::Protocol::Automation::MouseInteraction::Up:
- doMouseEvent(GDK_BUTTON_RELEASE, page.viewWidget(), viewPosition, gdkButton, state);
+ case MouseInteraction::Up:
+ doMouseEvent(GDK_BUTTON_RELEASE, page.viewWidget(), locationInView, gdkButton, state);
break;
- case Inspector::Protocol::Automation::MouseInteraction::SingleClick:
- doMouseEvent(GDK_BUTTON_PRESS, page.viewWidget(), viewPosition, gdkButton, state);
- doMouseEvent(GDK_BUTTON_RELEASE, page.viewWidget(), viewPosition, gdkButton, state);
+ case MouseInteraction::SingleClick:
+ doMouseEvent(GDK_BUTTON_PRESS, page.viewWidget(), locationInView, gdkButton, state);
+ doMouseEvent(GDK_BUTTON_RELEASE, page.viewWidget(), locationInView, gdkButton, state);
break;
- case Inspector::Protocol::Automation::MouseInteraction::DoubleClick:
- doMouseEvent(GDK_BUTTON_PRESS, page.viewWidget(), viewPosition, gdkButton, state);
- doMouseEvent(GDK_BUTTON_RELEASE, page.viewWidget(), viewPosition, gdkButton, state);
- doMouseEvent(GDK_BUTTON_PRESS, page.viewWidget(), viewPosition, gdkButton, state);
- doMouseEvent(GDK_BUTTON_RELEASE, page.viewWidget(), viewPosition, gdkButton, state);
+ case MouseInteraction::DoubleClick:
+ doMouseEvent(GDK_BUTTON_PRESS, page.viewWidget(), locationInView, gdkButton, state);
+ doMouseEvent(GDK_BUTTON_RELEASE, page.viewWidget(), locationInView, gdkButton, state);
+ doMouseEvent(GDK_BUTTON_PRESS, page.viewWidget(), locationInView, gdkButton, state);
+ doMouseEvent(GDK_BUTTON_RELEASE, page.viewWidget(), locationInView, gdkButton, state);
break;
}
}
@@ -280,20 +280,27 @@
return 0;
}
-void WebAutomationSession::platformSimulateKeyStroke(WebPageProxy& page, Inspector::Protocol::Automation::KeyboardInteractionType interaction, Inspector::Protocol::Automation::VirtualKey key)
+void WebAutomationSession::platformSimulateKeyboardInteraction(WebPageProxy& page, KeyboardInteraction interaction, std::optional<VirtualKey> virtualKey, std::optional<CharKey> charKey)
{
+ ASSERT(virtualKey.has_value() || charKey.has_value());
+
GdkModifierType updateState;
- auto keyCode = keyCodeForVirtualKey(key, updateState);
+ int keyCode;
+ if (virtualKey.has_value())
+ keyCode = keyCodeForVirtualKey(virtualKey.value(), updateState);
+ else
+ keyCode = gdk_unicode_to_keyval(g_utf8_get_char(&charKey.value()));
+
switch (interaction) {
- case Inspector::Protocol::Automation::KeyboardInteractionType::KeyPress:
+ case KeyboardInteraction::KeyPress:
m_currentModifiers |= updateState;
doKeyStrokeEvent(GDK_KEY_PRESS, page.viewWidget(), keyCode, m_currentModifiers);
break;
- case Inspector::Protocol::Automation::KeyboardInteractionType::KeyRelease:
+ case KeyboardInteraction::KeyRelease:
m_currentModifiers &= ~updateState;
doKeyStrokeEvent(GDK_KEY_RELEASE, page.viewWidget(), keyCode, m_currentModifiers);
break;
- case Inspector::Protocol::Automation::KeyboardInteractionType::InsertByKey:
+ case KeyboardInteraction::InsertByKey:
doKeyStrokeEvent(GDK_KEY_PRESS, page.viewWidget(), keyCode, m_currentModifiers, true);
break;
}
Modified: trunk/Source/WebKit/UIProcess/Automation/ios/WebAutomationSessionIOS.mm (230829 => 230830)
--- trunk/Source/WebKit/UIProcess/Automation/ios/WebAutomationSessionIOS.mm 2018-04-20 01:00:40 UTC (rev 230829)
+++ trunk/Source/WebKit/UIProcess/Automation/ios/WebAutomationSessionIOS.mm 2018-04-20 01:45:21 UTC (rev 230830)
@@ -65,38 +65,50 @@
#pragma mark Commands for Platform: 'iOS'
-void WebAutomationSession::platformSimulateKeyStroke(WebPageProxy& page, Inspector::Protocol::Automation::KeyboardInteractionType interaction, Inspector::Protocol::Automation::VirtualKey key)
+void WebAutomationSession::platformSimulateKeyboardInteraction(WebPageProxy& page, KeyboardInteraction interaction, std::optional<VirtualKey> virtualKey, std::optional<CharKey> charKey)
{
+ ASSERT(virtualKey.has_value() || charKey.has_value());
+
// The modifiers changed by the virtual key when it is pressed or released.
WebEventFlags changedModifiers = 0;
- // Figure out the effects of sticky modifiers.
- switch (key) {
- case Inspector::Protocol::Automation::VirtualKey::Shift:
- changedModifiers |= WebEventFlagMaskShift;
- break;
- case Inspector::Protocol::Automation::VirtualKey::Control:
- changedModifiers |= WebEventFlagMaskControl;
- break;
- case Inspector::Protocol::Automation::VirtualKey::Alternate:
- changedModifiers |= WebEventFlagMaskAlternate;
- break;
- case Inspector::Protocol::Automation::VirtualKey::Meta:
- // The 'meta' key does not exist on Apple keyboards and is usually
- // mapped to the Command key when using third-party keyboards.
- case Inspector::Protocol::Automation::VirtualKey::Command:
- changedModifiers |= WebEventFlagMaskCommand;
- break;
- default:
- break;
- }
-
// UIKit does not send key codes for virtual keys even for a hardware keyboard.
// Instead, it sends single unichars and WebCore maps these to "windows" key codes.
// Synthesize a single unichar such that the correct key code is inferred.
- std::optional<unichar> charCode = charCodeForVirtualKey(key);
- std::optional<unichar> charCodeIgnoringModifiers = charCodeIgnoringModifiersForVirtualKey(key);
+ std::optional<unichar> charCode;
+ std::optional<unichar> charCodeIgnoringModifiers;
+ // Figure out the effects of sticky modifiers.
+ if (virtualKey.has_value()) {
+ charCode = charCodeForVirtualKey(virtualKey.value());
+ charCodeIgnoringModifiers = charCodeIgnoringModifiersForVirtualKey(virtualKey.value());
+
+ switch (virtualKey.value()) {
+ case VirtualKey::Shift:
+ changedModifiers |= WebEventFlagMaskShift;
+ break;
+ case VirtualKey::Control:
+ changedModifiers |= WebEventFlagMaskControl;
+ break;
+ case VirtualKey::Alternate:
+ changedModifiers |= WebEventFlagMaskAlternate;
+ break;
+ case VirtualKey::Meta:
+ // The 'meta' key does not exist on Apple keyboards and is usually
+ // mapped to the Command key when using third-party keyboards.
+ case VirtualKey::Command:
+ changedModifiers |= WebEventFlagMaskCommand;
+ break;
+ default:
+ break;
+ }
+ }
+
+ if (charKey.has_value()) {
+ charCode = (unichar)charKey.value();
+ charCodeIgnoringModifiers = (unichar)charKey.value();
+ }
+
// FIXME: consider using UIKit SPI to normalize 'characters', i.e., changing * to Shift-8,
// and passing that in to charactersIgnoringModifiers. This is probably not worth the trouble
// unless it causes an actual behavioral difference.
@@ -115,19 +127,19 @@
auto eventsToBeSent = adoptNS([[NSMutableArray alloc] init]);
switch (interaction) {
- case Inspector::Protocol::Automation::KeyboardInteractionType::KeyPress: {
+ case KeyboardInteraction::KeyPress: {
m_currentModifiers |= changedModifiers;
[eventsToBeSent addObject:[[[::WebEvent alloc] initWithKeyEventType:WebEventKeyDown timeStamp:CFAbsoluteTimeGetCurrent() characters:characters charactersIgnoringModifiers:unmodifiedCharacters modifiers:m_currentModifiers isRepeating:NO withFlags:inputFlags withInputManagerHint:nil keyCode:keyCode isTabKey:isTabKey] autorelease]];
break;
}
- case Inspector::Protocol::Automation::KeyboardInteractionType::KeyRelease: {
+ case KeyboardInteraction::KeyRelease: {
m_currentModifiers &= ~changedModifiers;
[eventsToBeSent addObject:[[[::WebEvent alloc] initWithKeyEventType:WebEventKeyUp timeStamp:CFAbsoluteTimeGetCurrent() characters:characters charactersIgnoringModifiers:unmodifiedCharacters modifiers:m_currentModifiers isRepeating:NO withFlags:inputFlags withInputManagerHint:nil keyCode:keyCode isTabKey:isTabKey] autorelease]];
break;
}
- case Inspector::Protocol::Automation::KeyboardInteractionType::InsertByKey: {
+ case KeyboardInteraction::InsertByKey: {
// Modifiers only change with KeyPress or KeyRelease, this code path is for single characters.
[eventsToBeSent addObject:[[[::WebEvent alloc] initWithKeyEventType:WebEventKeyDown timeStamp:CFAbsoluteTimeGetCurrent() characters:characters charactersIgnoringModifiers:unmodifiedCharacters modifiers:m_currentModifiers isRepeating:NO withFlags:inputFlags withInputManagerHint:nil keyCode:keyCode isTabKey:isTabKey] autorelease]];
[eventsToBeSent addObject:[[[::WebEvent alloc] initWithKeyEventType:WebEventKeyUp timeStamp:CFAbsoluteTimeGetCurrent() characters:characters charactersIgnoringModifiers:unmodifiedCharacters modifiers:m_currentModifiers isRepeating:NO withFlags:inputFlags withInputManagerHint:nil keyCode:keyCode isTabKey:isTabKey] autorelease]];
Modified: trunk/Source/WebKit/UIProcess/Automation/mac/WebAutomationSessionMac.mm (230829 => 230830)
--- trunk/Source/WebKit/UIProcess/Automation/mac/WebAutomationSessionMac.mm 2018-04-20 01:00:40 UTC (rev 230829)
+++ trunk/Source/WebKit/UIProcess/Automation/mac/WebAutomationSessionMac.mm 2018-04-20 01:45:21 UTC (rev 230830)
@@ -118,11 +118,11 @@
#pragma mark Platform-dependent Implementations
-void WebAutomationSession::platformSimulateMouseInteraction(WebPageProxy& page, const WebCore::IntPoint& viewPosition, Inspector::Protocol::Automation::MouseInteraction interaction, Inspector::Protocol::Automation::MouseButton button, WebEvent::Modifiers keyModifiers)
+void WebAutomationSession::platformSimulateMouseInteraction(WebPageProxy& page, MouseInteraction interaction, WebMouseEvent::Button button, const WebCore::IntPoint& locationInView, WebEvent::Modifiers keyModifiers)
{
IntRect windowRect;
- page.rootViewToWindow(IntRect(viewPosition, IntSize()), windowRect);
- IntPoint windowPosition = windowRect.location();
+ page.rootViewToWindow(IntRect(locationInView, IntSize()), windowRect);
+ IntPoint locationInWindow = windowRect.location();
NSEventModifierFlags modifiers = 0;
if (keyModifiers & WebEvent::MetaKey)
@@ -144,20 +144,20 @@
NSEventType dragEventType = (NSEventType)0;
NSEventType upEventType = (NSEventType)0;
switch (button) {
- case Inspector::Protocol::Automation::MouseButton::None:
+ case WebMouseEvent::NoButton:
downEventType = upEventType = dragEventType = NSEventTypeMouseMoved;
break;
- case Inspector::Protocol::Automation::MouseButton::Left:
+ case WebMouseEvent::LeftButton:
downEventType = NSEventTypeLeftMouseDown;
dragEventType = NSEventTypeLeftMouseDragged;
upEventType = NSEventTypeLeftMouseUp;
break;
- case Inspector::Protocol::Automation::MouseButton::Middle:
+ case WebMouseEvent::MiddleButton:
downEventType = NSEventTypeOtherMouseDown;
dragEventType = NSEventTypeLeftMouseDragged;
upEventType = NSEventTypeOtherMouseUp;
break;
- case Inspector::Protocol::Automation::MouseButton::Right:
+ case WebMouseEvent::RightButton:
downEventType = NSEventTypeRightMouseDown;
upEventType = NSEventTypeRightMouseUp;
break;
@@ -168,56 +168,56 @@
NSInteger eventNumber = synthesizedMouseEventMagicEventNumber;
switch (interaction) {
- case Inspector::Protocol::Automation::MouseInteraction::Move:
+ case MouseInteraction::Move:
ASSERT(dragEventType);
- [eventsToBeSent addObject:[NSEvent mouseEventWithType:dragEventType location:windowPosition modifierFlags:modifiers timestamp:timestamp windowNumber:windowNumber context:nil eventNumber:eventNumber clickCount:0 pressure:0.0f]];
+ [eventsToBeSent addObject:[NSEvent mouseEventWithType:dragEventType location:locationInWindow modifierFlags:modifiers timestamp:timestamp windowNumber:windowNumber context:nil eventNumber:eventNumber clickCount:0 pressure:0.0f]];
break;
- case Inspector::Protocol::Automation::MouseInteraction::Down:
+ case MouseInteraction::Down:
ASSERT(downEventType);
// Hard-code the click count to one, since clients don't expect successive simulated
// down/up events to be potentially counted as a double click event.
- [eventsToBeSent addObject:[NSEvent mouseEventWithType:downEventType location:windowPosition modifierFlags:modifiers timestamp:timestamp windowNumber:windowNumber context:nil eventNumber:eventNumber clickCount:1 pressure:WebCore::ForceAtClick]];
+ [eventsToBeSent addObject:[NSEvent mouseEventWithType:downEventType location:locationInWindow modifierFlags:modifiers timestamp:timestamp windowNumber:windowNumber context:nil eventNumber:eventNumber clickCount:1 pressure:WebCore::ForceAtClick]];
break;
- case Inspector::Protocol::Automation::MouseInteraction::Up:
+ case MouseInteraction::Up:
ASSERT(upEventType);
// Hard-code the click count to one, since clients don't expect successive simulated
// down/up events to be potentially counted as a double click event.
- [eventsToBeSent addObject:[NSEvent mouseEventWithType:upEventType location:windowPosition modifierFlags:modifiers timestamp:timestamp windowNumber:windowNumber context:nil eventNumber:eventNumber clickCount:1 pressure:0.0f]];
+ [eventsToBeSent addObject:[NSEvent mouseEventWithType:upEventType location:locationInWindow modifierFlags:modifiers timestamp:timestamp windowNumber:windowNumber context:nil eventNumber:eventNumber clickCount:1 pressure:0.0f]];
break;
- case Inspector::Protocol::Automation::MouseInteraction::SingleClick:
+ case MouseInteraction::SingleClick:
ASSERT(upEventType);
ASSERT(downEventType);
// Send separate down and up events. WebCore will see this as a single-click event.
- [eventsToBeSent addObject:[NSEvent mouseEventWithType:downEventType location:windowPosition modifierFlags:modifiers timestamp:timestamp windowNumber:windowNumber context:nil eventNumber:eventNumber clickCount:1 pressure:WebCore::ForceAtClick]];
- [eventsToBeSent addObject:[NSEvent mouseEventWithType:upEventType location:windowPosition modifierFlags:modifiers timestamp:timestamp windowNumber:windowNumber context:nil eventNumber:eventNumber clickCount:1 pressure:0.0f]];
+ [eventsToBeSent addObject:[NSEvent mouseEventWithType:downEventType location:locationInWindow modifierFlags:modifiers timestamp:timestamp windowNumber:windowNumber context:nil eventNumber:eventNumber clickCount:1 pressure:WebCore::ForceAtClick]];
+ [eventsToBeSent addObject:[NSEvent mouseEventWithType:upEventType location:locationInWindow modifierFlags:modifiers timestamp:timestamp windowNumber:windowNumber context:nil eventNumber:eventNumber clickCount:1 pressure:0.0f]];
break;
- case Inspector::Protocol::Automation::MouseInteraction::DoubleClick:
+ case MouseInteraction::DoubleClick:
ASSERT(upEventType);
ASSERT(downEventType);
// Send multiple down and up events with proper click count.
// WebCore will see this as a single-click event then double-click event.
- [eventsToBeSent addObject:[NSEvent mouseEventWithType:downEventType location:windowPosition modifierFlags:modifiers timestamp:timestamp windowNumber:windowNumber context:nil eventNumber:eventNumber clickCount:1 pressure:WebCore::ForceAtClick]];
- [eventsToBeSent addObject:[NSEvent mouseEventWithType:upEventType location:windowPosition modifierFlags:modifiers timestamp:timestamp windowNumber:windowNumber context:nil eventNumber:eventNumber clickCount:1 pressure:0.0f]];
- [eventsToBeSent addObject:[NSEvent mouseEventWithType:downEventType location:windowPosition modifierFlags:modifiers timestamp:timestamp windowNumber:windowNumber context:nil eventNumber:eventNumber clickCount:2 pressure:WebCore::ForceAtClick]];
- [eventsToBeSent addObject:[NSEvent mouseEventWithType:upEventType location:windowPosition modifierFlags:modifiers timestamp:timestamp windowNumber:windowNumber context:nil eventNumber:eventNumber clickCount:2 pressure:0.0f]];
+ [eventsToBeSent addObject:[NSEvent mouseEventWithType:downEventType location:locationInWindow modifierFlags:modifiers timestamp:timestamp windowNumber:windowNumber context:nil eventNumber:eventNumber clickCount:1 pressure:WebCore::ForceAtClick]];
+ [eventsToBeSent addObject:[NSEvent mouseEventWithType:upEventType location:locationInWindow modifierFlags:modifiers timestamp:timestamp windowNumber:windowNumber context:nil eventNumber:eventNumber clickCount:1 pressure:0.0f]];
+ [eventsToBeSent addObject:[NSEvent mouseEventWithType:downEventType location:locationInWindow modifierFlags:modifiers timestamp:timestamp windowNumber:windowNumber context:nil eventNumber:eventNumber clickCount:2 pressure:WebCore::ForceAtClick]];
+ [eventsToBeSent addObject:[NSEvent mouseEventWithType:upEventType location:locationInWindow modifierFlags:modifiers timestamp:timestamp windowNumber:windowNumber context:nil eventNumber:eventNumber clickCount:2 pressure:0.0f]];
}
sendSynthesizedEventsToPage(page, eventsToBeSent.get());
}
-static bool keyHasStickyModifier(Inspector::Protocol::Automation::VirtualKey key)
+static bool virtualKeyHasStickyModifier(VirtualKey key)
{
// Returns whether the key's modifier flags should affect other events while pressed down.
switch (key) {
- case Inspector::Protocol::Automation::VirtualKey::Shift:
- case Inspector::Protocol::Automation::VirtualKey::Control:
- case Inspector::Protocol::Automation::VirtualKey::Alternate:
- case Inspector::Protocol::Automation::VirtualKey::Meta:
- case Inspector::Protocol::Automation::VirtualKey::Command:
+ case VirtualKey::Shift:
+ case VirtualKey::Control:
+ case VirtualKey::Alternate:
+ case VirtualKey::Meta:
+ case VirtualKey::Command:
return true;
default:
@@ -225,201 +225,201 @@
}
}
-static int keyCodeForVirtualKey(Inspector::Protocol::Automation::VirtualKey key)
+static int keyCodeForVirtualKey(VirtualKey key)
{
// The likely keyCode for the virtual key as defined in <HIToolbox/Events.h>.
switch (key) {
- case Inspector::Protocol::Automation::VirtualKey::Shift:
+ case VirtualKey::Shift:
return kVK_Shift;
- case Inspector::Protocol::Automation::VirtualKey::Control:
+ case VirtualKey::Control:
return kVK_Control;
- case Inspector::Protocol::Automation::VirtualKey::Alternate:
+ case VirtualKey::Alternate:
return kVK_Option;
- case Inspector::Protocol::Automation::VirtualKey::Meta:
+ case VirtualKey::Meta:
// The 'meta' key does not exist on Apple keyboards and is usually
// mapped to the Command key when using third-party keyboards.
- case Inspector::Protocol::Automation::VirtualKey::Command:
+ case VirtualKey::Command:
return kVK_Command;
- case Inspector::Protocol::Automation::VirtualKey::Help:
+ case VirtualKey::Help:
return kVK_Help;
- case Inspector::Protocol::Automation::VirtualKey::Backspace:
+ case VirtualKey::Backspace:
return kVK_Delete;
- case Inspector::Protocol::Automation::VirtualKey::Tab:
+ case VirtualKey::Tab:
return kVK_Tab;
- case Inspector::Protocol::Automation::VirtualKey::Clear:
+ case VirtualKey::Clear:
return kVK_ANSI_KeypadClear;
- case Inspector::Protocol::Automation::VirtualKey::Enter:
+ case VirtualKey::Enter:
return kVK_ANSI_KeypadEnter;
- case Inspector::Protocol::Automation::VirtualKey::Pause:
+ case VirtualKey::Pause:
// The 'pause' key does not exist on Apple keyboards and has no keyCode.
// The semantics are unclear so just abort and do nothing.
return 0;
- case Inspector::Protocol::Automation::VirtualKey::Cancel:
+ case VirtualKey::Cancel:
// The 'cancel' key does not exist on Apple keyboards and has no keyCode.
// According to the internet its functionality is similar to 'Escape'.
- case Inspector::Protocol::Automation::VirtualKey::Escape:
+ case VirtualKey::Escape:
return kVK_Escape;
- case Inspector::Protocol::Automation::VirtualKey::PageUp:
+ case VirtualKey::PageUp:
return kVK_PageUp;
- case Inspector::Protocol::Automation::VirtualKey::PageDown:
+ case VirtualKey::PageDown:
return kVK_PageDown;
- case Inspector::Protocol::Automation::VirtualKey::End:
+ case VirtualKey::End:
return kVK_End;
- case Inspector::Protocol::Automation::VirtualKey::Home:
+ case VirtualKey::Home:
return kVK_Home;
- case Inspector::Protocol::Automation::VirtualKey::LeftArrow:
+ case VirtualKey::LeftArrow:
return kVK_LeftArrow;
- case Inspector::Protocol::Automation::VirtualKey::UpArrow:
+ case VirtualKey::UpArrow:
return kVK_UpArrow;
- case Inspector::Protocol::Automation::VirtualKey::RightArrow:
+ case VirtualKey::RightArrow:
return kVK_RightArrow;
- case Inspector::Protocol::Automation::VirtualKey::DownArrow:
+ case VirtualKey::DownArrow:
return kVK_DownArrow;
- case Inspector::Protocol::Automation::VirtualKey::Insert:
+ case VirtualKey::Insert:
// The 'insert' key does not exist on Apple keyboards and has no keyCode.
// The semantics are unclear so just abort and do nothing.
return 0;
- case Inspector::Protocol::Automation::VirtualKey::Delete:
+ case VirtualKey::Delete:
return kVK_ForwardDelete;
- case Inspector::Protocol::Automation::VirtualKey::Space:
+ case VirtualKey::Space:
return kVK_Space;
- case Inspector::Protocol::Automation::VirtualKey::Semicolon:
+ case VirtualKey::Semicolon:
return kVK_ANSI_Semicolon;
- case Inspector::Protocol::Automation::VirtualKey::Equals:
+ case VirtualKey::Equals:
return kVK_ANSI_Equal;
- case Inspector::Protocol::Automation::VirtualKey::Return:
+ case VirtualKey::Return:
return kVK_Return;
- case Inspector::Protocol::Automation::VirtualKey::NumberPad0:
+ case VirtualKey::NumberPad0:
return kVK_ANSI_Keypad0;
- case Inspector::Protocol::Automation::VirtualKey::NumberPad1:
+ case VirtualKey::NumberPad1:
return kVK_ANSI_Keypad1;
- case Inspector::Protocol::Automation::VirtualKey::NumberPad2:
+ case VirtualKey::NumberPad2:
return kVK_ANSI_Keypad2;
- case Inspector::Protocol::Automation::VirtualKey::NumberPad3:
+ case VirtualKey::NumberPad3:
return kVK_ANSI_Keypad3;
- case Inspector::Protocol::Automation::VirtualKey::NumberPad4:
+ case VirtualKey::NumberPad4:
return kVK_ANSI_Keypad4;
- case Inspector::Protocol::Automation::VirtualKey::NumberPad5:
+ case VirtualKey::NumberPad5:
return kVK_ANSI_Keypad5;
- case Inspector::Protocol::Automation::VirtualKey::NumberPad6:
+ case VirtualKey::NumberPad6:
return kVK_ANSI_Keypad6;
- case Inspector::Protocol::Automation::VirtualKey::NumberPad7:
+ case VirtualKey::NumberPad7:
return kVK_ANSI_Keypad7;
- case Inspector::Protocol::Automation::VirtualKey::NumberPad8:
+ case VirtualKey::NumberPad8:
return kVK_ANSI_Keypad8;
- case Inspector::Protocol::Automation::VirtualKey::NumberPad9:
+ case VirtualKey::NumberPad9:
return kVK_ANSI_Keypad9;
- case Inspector::Protocol::Automation::VirtualKey::NumberPadMultiply:
+ case VirtualKey::NumberPadMultiply:
return kVK_ANSI_KeypadMultiply;
- case Inspector::Protocol::Automation::VirtualKey::NumberPadAdd:
+ case VirtualKey::NumberPadAdd:
return kVK_ANSI_KeypadPlus;
- case Inspector::Protocol::Automation::VirtualKey::NumberPadSubtract:
+ case VirtualKey::NumberPadSubtract:
return kVK_ANSI_KeypadMinus;
- case Inspector::Protocol::Automation::VirtualKey::NumberPadSeparator:
+ case VirtualKey::NumberPadSeparator:
// The 'Separator' key is only present on a few international keyboards.
// It is usually mapped to the same character as Decimal ('.' or ',').
FALLTHROUGH;
- case Inspector::Protocol::Automation::VirtualKey::NumberPadDecimal:
+ case VirtualKey::NumberPadDecimal:
return kVK_ANSI_KeypadDecimal;
// FIXME: this might be locale-dependent. See the above comment.
- case Inspector::Protocol::Automation::VirtualKey::NumberPadDivide:
+ case VirtualKey::NumberPadDivide:
return kVK_ANSI_KeypadDivide;
- case Inspector::Protocol::Automation::VirtualKey::Function1:
+ case VirtualKey::Function1:
return kVK_F1;
- case Inspector::Protocol::Automation::VirtualKey::Function2:
+ case VirtualKey::Function2:
return kVK_F2;
- case Inspector::Protocol::Automation::VirtualKey::Function3:
+ case VirtualKey::Function3:
return kVK_F3;
- case Inspector::Protocol::Automation::VirtualKey::Function4:
+ case VirtualKey::Function4:
return kVK_F4;
- case Inspector::Protocol::Automation::VirtualKey::Function5:
+ case VirtualKey::Function5:
return kVK_F5;
- case Inspector::Protocol::Automation::VirtualKey::Function6:
+ case VirtualKey::Function6:
return kVK_F6;
- case Inspector::Protocol::Automation::VirtualKey::Function7:
+ case VirtualKey::Function7:
return kVK_F7;
- case Inspector::Protocol::Automation::VirtualKey::Function8:
+ case VirtualKey::Function8:
return kVK_F8;
- case Inspector::Protocol::Automation::VirtualKey::Function9:
+ case VirtualKey::Function9:
return kVK_F9;
- case Inspector::Protocol::Automation::VirtualKey::Function10:
+ case VirtualKey::Function10:
return kVK_F10;
- case Inspector::Protocol::Automation::VirtualKey::Function11:
+ case VirtualKey::Function11:
return kVK_F11;
- case Inspector::Protocol::Automation::VirtualKey::Function12:
+ case VirtualKey::Function12:
return kVK_F12;
}
}
-static NSEventModifierFlags eventModifierFlagsForVirtualKey(Inspector::Protocol::Automation::VirtualKey key)
+static NSEventModifierFlags eventModifierFlagsForVirtualKey(VirtualKey key)
{
// Computes the modifiers changed by the virtual key when it is pressed or released.
// The mapping from keys to modifiers is specified in the documentation for NSEvent.
switch (key) {
- case Inspector::Protocol::Automation::VirtualKey::Shift:
+ case VirtualKey::Shift:
return NSEventModifierFlagShift;
- case Inspector::Protocol::Automation::VirtualKey::Control:
+ case VirtualKey::Control:
return NSEventModifierFlagControl;
- case Inspector::Protocol::Automation::VirtualKey::Alternate:
+ case VirtualKey::Alternate:
return NSEventModifierFlagOption;
- case Inspector::Protocol::Automation::VirtualKey::Meta:
+ case VirtualKey::Meta:
// The 'meta' key does not exist on Apple keyboards and is usually
// mapped to the Command key when using third-party keyboards.
- case Inspector::Protocol::Automation::VirtualKey::Command:
+ case VirtualKey::Command:
return NSEventModifierFlagCommand;
- case Inspector::Protocol::Automation::VirtualKey::Help:
+ case VirtualKey::Help:
return NSEventModifierFlagHelp | NSEventModifierFlagFunction;
- case Inspector::Protocol::Automation::VirtualKey::PageUp:
- case Inspector::Protocol::Automation::VirtualKey::PageDown:
- case Inspector::Protocol::Automation::VirtualKey::End:
- case Inspector::Protocol::Automation::VirtualKey::Home:
+ case VirtualKey::PageUp:
+ case VirtualKey::PageDown:
+ case VirtualKey::End:
+ case VirtualKey::Home:
return NSEventModifierFlagFunction;
- case Inspector::Protocol::Automation::VirtualKey::LeftArrow:
- case Inspector::Protocol::Automation::VirtualKey::UpArrow:
- case Inspector::Protocol::Automation::VirtualKey::RightArrow:
- case Inspector::Protocol::Automation::VirtualKey::DownArrow:
+ case VirtualKey::LeftArrow:
+ case VirtualKey::UpArrow:
+ case VirtualKey::RightArrow:
+ case VirtualKey::DownArrow:
return NSEventModifierFlagNumericPad | NSEventModifierFlagFunction;
- case Inspector::Protocol::Automation::VirtualKey::Delete:
+ case VirtualKey::Delete:
return NSEventModifierFlagFunction;
- case Inspector::Protocol::Automation::VirtualKey::Clear:
- case Inspector::Protocol::Automation::VirtualKey::NumberPad0:
- case Inspector::Protocol::Automation::VirtualKey::NumberPad1:
- case Inspector::Protocol::Automation::VirtualKey::NumberPad2:
- case Inspector::Protocol::Automation::VirtualKey::NumberPad3:
- case Inspector::Protocol::Automation::VirtualKey::NumberPad4:
- case Inspector::Protocol::Automation::VirtualKey::NumberPad5:
- case Inspector::Protocol::Automation::VirtualKey::NumberPad6:
- case Inspector::Protocol::Automation::VirtualKey::NumberPad7:
- case Inspector::Protocol::Automation::VirtualKey::NumberPad8:
- case Inspector::Protocol::Automation::VirtualKey::NumberPad9:
- case Inspector::Protocol::Automation::VirtualKey::NumberPadMultiply:
- case Inspector::Protocol::Automation::VirtualKey::NumberPadAdd:
- case Inspector::Protocol::Automation::VirtualKey::NumberPadSubtract:
- case Inspector::Protocol::Automation::VirtualKey::NumberPadSeparator:
- case Inspector::Protocol::Automation::VirtualKey::NumberPadDecimal:
- case Inspector::Protocol::Automation::VirtualKey::NumberPadDivide:
+ case VirtualKey::Clear:
+ case VirtualKey::NumberPad0:
+ case VirtualKey::NumberPad1:
+ case VirtualKey::NumberPad2:
+ case VirtualKey::NumberPad3:
+ case VirtualKey::NumberPad4:
+ case VirtualKey::NumberPad5:
+ case VirtualKey::NumberPad6:
+ case VirtualKey::NumberPad7:
+ case VirtualKey::NumberPad8:
+ case VirtualKey::NumberPad9:
+ case VirtualKey::NumberPadMultiply:
+ case VirtualKey::NumberPadAdd:
+ case VirtualKey::NumberPadSubtract:
+ case VirtualKey::NumberPadSeparator:
+ case VirtualKey::NumberPadDecimal:
+ case VirtualKey::NumberPadDivide:
return NSEventModifierFlagNumericPad;
- case Inspector::Protocol::Automation::VirtualKey::Function1:
- case Inspector::Protocol::Automation::VirtualKey::Function2:
- case Inspector::Protocol::Automation::VirtualKey::Function3:
- case Inspector::Protocol::Automation::VirtualKey::Function4:
- case Inspector::Protocol::Automation::VirtualKey::Function5:
- case Inspector::Protocol::Automation::VirtualKey::Function6:
- case Inspector::Protocol::Automation::VirtualKey::Function7:
- case Inspector::Protocol::Automation::VirtualKey::Function8:
- case Inspector::Protocol::Automation::VirtualKey::Function9:
- case Inspector::Protocol::Automation::VirtualKey::Function10:
- case Inspector::Protocol::Automation::VirtualKey::Function11:
- case Inspector::Protocol::Automation::VirtualKey::Function12:
+ case VirtualKey::Function1:
+ case VirtualKey::Function2:
+ case VirtualKey::Function3:
+ case VirtualKey::Function4:
+ case VirtualKey::Function5:
+ case VirtualKey::Function6:
+ case VirtualKey::Function7:
+ case VirtualKey::Function8:
+ case VirtualKey::Function9:
+ case VirtualKey::Function10:
+ case VirtualKey::Function11:
+ case VirtualKey::Function12:
return NSEventModifierFlagFunction;
default:
@@ -427,19 +427,34 @@
}
}
-void WebAutomationSession::platformSimulateKeyStroke(WebPageProxy& page, Inspector::Protocol::Automation::KeyboardInteractionType interaction, Inspector::Protocol::Automation::VirtualKey key)
+void WebAutomationSession::platformSimulateKeyboardInteraction(WebPageProxy& page, KeyboardInteraction interaction, std::optional<VirtualKey> virtualKey, std::optional<CharKey> charKey)
{
// FIXME: this function and the Automation protocol enum should probably adopt key names
// from W3C UIEvents standard. For more details: https://w3c.github.io/uievents-code/
- bool isStickyModifier = keyHasStickyModifier(key);
- NSEventModifierFlags changedModifiers = eventModifierFlagsForVirtualKey(key);
- int keyCode = keyCodeForVirtualKey(key);
+ ASSERT(virtualKey.has_value() || charKey.has_value());
+ bool isStickyModifier = false;
+ NSEventModifierFlags changedModifiers = 0;
+ int keyCode;
+ std::optional<unichar> charCode;
+ std::optional<unichar> charCodeIgnoringModifiers;
+
+ if (virtualKey.has_value()) {
+ isStickyModifier = virtualKeyHasStickyModifier(virtualKey.value());
+ changedModifiers = eventModifierFlagsForVirtualKey(virtualKey.value());
+ keyCode = keyCodeForVirtualKey(virtualKey.value());
+ charCode = charCodeForVirtualKey(virtualKey.value());
+ charCodeIgnoringModifiers = charCodeIgnoringModifiersForVirtualKey(virtualKey.value());
+ }
+
+ if (charKey.has_value()) {
+ charCode = (unichar)charKey.value();
+ charCodeIgnoringModifiers = (unichar)charKey.value();
+ }
+
// FIXME: consider using AppKit SPI to normalize 'characters', i.e., changing * to Shift-8,
// and passing that in to charactersIgnoringModifiers. We could hardcode this for ASCII if needed.
- std::optional<unichar> charCode = charCodeForVirtualKey(key);
- std::optional<unichar> charCodeIgnoringModifiers = charCodeIgnoringModifiersForVirtualKey(key);
NSString *characters = charCode ? [NSString stringWithCharacters:&charCode.value() length:1] : nil;
NSString *unmodifiedCharacters = charCodeIgnoringModifiers ? [NSString stringWithCharacters:&charCodeIgnoringModifiers.value() length:1] : nil;
@@ -452,19 +467,19 @@
NSPoint eventPosition = NSMakePoint(0, window.frame.size.height);
switch (interaction) {
- case Inspector::Protocol::Automation::KeyboardInteractionType::KeyPress: {
+ case KeyboardInteraction::KeyPress: {
NSEventType eventType = isStickyModifier ? NSEventTypeFlagsChanged : NSEventTypeKeyDown;
m_currentModifiers |= changedModifiers;
[eventsToBeSent addObject:[NSEvent keyEventWithType:eventType location:eventPosition modifierFlags:m_currentModifiers timestamp:timestamp windowNumber:windowNumber context:nil characters:characters charactersIgnoringModifiers:unmodifiedCharacters isARepeat:NO keyCode:keyCode]];
break;
}
- case Inspector::Protocol::Automation::KeyboardInteractionType::KeyRelease: {
+ case KeyboardInteraction::KeyRelease: {
NSEventType eventType = isStickyModifier ? NSEventTypeFlagsChanged : NSEventTypeKeyUp;
m_currentModifiers &= ~changedModifiers;
[eventsToBeSent addObject:[NSEvent keyEventWithType:eventType location:eventPosition modifierFlags:m_currentModifiers timestamp:timestamp windowNumber:windowNumber context:nil characters:characters charactersIgnoringModifiers:unmodifiedCharacters isARepeat:NO keyCode:keyCode]];
break;
}
- case Inspector::Protocol::Automation::KeyboardInteractionType::InsertByKey: {
+ case KeyboardInteraction::InsertByKey: {
// Sticky modifiers should either be 'KeyPress' or 'KeyRelease'.
ASSERT(!isStickyModifier);
if (isStickyModifier)
@@ -551,7 +566,7 @@
[eventsToBeSent addObject:[NSEvent keyEventWithType:NSEventTypeKeyDown location:eventPosition modifierFlags:modifiersForKeystroke timestamp:timestamp windowNumber:windowNumber context:nil characters:substring charactersIgnoringModifiers:substring isARepeat:NO keyCode:0]];
[eventsToBeSent addObject:[NSEvent keyEventWithType:NSEventTypeKeyUp location:eventPosition modifierFlags:modifiersForKeystroke timestamp:timestamp windowNumber:windowNumber context:nil characters:substring charactersIgnoringModifiers:substring isARepeat:NO keyCode:0]];
-
+
if (shouldPressShift)
[eventsToBeSent addObject:[NSEvent keyEventWithType:NSEventTypeFlagsChanged location:eventPosition modifierFlags:m_currentModifiers timestamp:timestamp windowNumber:windowNumber context:nil characters:@"" charactersIgnoringModifiers:@"" isARepeat:NO keyCode:kVK_Shift]];
}];
Modified: trunk/Source/WebKit/WebKit.xcodeproj/project.pbxproj (230829 => 230830)
--- trunk/Source/WebKit/WebKit.xcodeproj/project.pbxproj 2018-04-20 01:00:40 UTC (rev 230829)
+++ trunk/Source/WebKit/WebKit.xcodeproj/project.pbxproj 2018-04-20 01:45:21 UTC (rev 230830)
@@ -1529,6 +1529,8 @@
9946EF861E7B027000541E79 /* WebAutomationSessionIOS.mm in Sources */ = {isa = PBXBuildFile; fileRef = 9946EF851E7B026600541E79 /* WebAutomationSessionIOS.mm */; };
994BADF31F7D781100B571E7 /* WKInspectorViewController.mm in Sources */ = {isa = PBXBuildFile; fileRef = 994BADF21F7D77EB00B571E7 /* WKInspectorViewController.mm */; };
994BADF41F7D781400B571E7 /* WKInspectorViewController.h in Headers */ = {isa = PBXBuildFile; fileRef = 994BADF11F7D77EA00B571E7 /* WKInspectorViewController.h */; };
+ 995226D6207D184600F78420 /* SimulatedInputDispatcher.h in Headers */ = {isa = PBXBuildFile; fileRef = 995226D4207D184500F78420 /* SimulatedInputDispatcher.h */; };
+ 995226D7207D184600F78420 /* SimulatedInputDispatcher.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 995226D5207D184600F78420 /* SimulatedInputDispatcher.cpp */; };
9955A6EC1C7980C200EB6A93 /* WebAutomationSession.h in Headers */ = {isa = PBXBuildFile; fileRef = 9955A6EB1C7980BB00EB6A93 /* WebAutomationSession.h */; };
9955A6ED1C7980CA00EB6A93 /* WebAutomationSession.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 9955A6EA1C7980BB00EB6A93 /* WebAutomationSession.cpp */; };
9955A6EF1C79810800EB6A93 /* Automation.json in Headers */ = {isa = PBXBuildFile; fileRef = 9955A6E91C7980BB00EB6A93 /* Automation.json */; settings = {ATTRIBUTES = (Private, ); }; };
@@ -4019,6 +4021,8 @@
9946EF851E7B026600541E79 /* WebAutomationSessionIOS.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = WebAutomationSessionIOS.mm; sourceTree = "<group>"; };
994BADF11F7D77EA00B571E7 /* WKInspectorViewController.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WKInspectorViewController.h; sourceTree = "<group>"; };
994BADF21F7D77EB00B571E7 /* WKInspectorViewController.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = WKInspectorViewController.mm; sourceTree = "<group>"; };
+ 995226D4207D184500F78420 /* SimulatedInputDispatcher.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = SimulatedInputDispatcher.h; sourceTree = "<group>"; };
+ 995226D5207D184600F78420 /* SimulatedInputDispatcher.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = SimulatedInputDispatcher.cpp; sourceTree = "<group>"; };
9955A6E91C7980BB00EB6A93 /* Automation.json */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.json; path = Automation.json; sourceTree = "<group>"; };
9955A6EA1C7980BB00EB6A93 /* WebAutomationSession.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WebAutomationSession.cpp; sourceTree = "<group>"; };
9955A6EB1C7980BB00EB6A93 /* WebAutomationSession.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WebAutomationSession.h; sourceTree = "<group>"; };
@@ -6970,6 +6974,8 @@
9946EF821E7B023D00541E79 /* ios */,
99C3AE221DAD8E1400AF5C16 /* mac */,
9955A6E91C7980BB00EB6A93 /* Automation.json */,
+ 995226D5207D184600F78420 /* SimulatedInputDispatcher.cpp */,
+ 995226D4207D184500F78420 /* SimulatedInputDispatcher.h */,
9955A6EA1C7980BB00EB6A93 /* WebAutomationSession.cpp */,
9955A6EB1C7980BB00EB6A93 /* WebAutomationSession.h */,
1C0A19591C9006EA00FE0EBB /* WebAutomationSession.messages.in */,
@@ -9174,6 +9180,7 @@
8313F7EC1F7DAE0800B944EB /* SharedStringHashStore.h in Headers */,
8313F7EE1F7DAE0800B944EB /* SharedStringHashTable.h in Headers */,
83F9644E1FA0F76E00C47750 /* SharedStringHashTableReadOnly.h in Headers */,
+ 995226D6207D184600F78420 /* SimulatedInputDispatcher.h in Headers */,
2DAF06D618BD1A470081CEB1 /* SmartMagnificationController.h in Headers */,
2DE6943E18BD2A68005C15E5 /* SmartMagnificationControllerMessages.h in Headers */,
5272B28B1406985D0096A5D0 /* StatisticsData.h in Headers */,
@@ -10880,6 +10887,7 @@
8313F7EB1F7DAE0800B944EB /* SharedStringHashStore.cpp in Sources */,
8313F7ED1F7DAE0800B944EB /* SharedStringHashTable.cpp in Sources */,
83F9644D1FA0F76E00C47750 /* SharedStringHashTableReadOnly.cpp in Sources */,
+ 995226D7207D184600F78420 /* SimulatedInputDispatcher.cpp in Sources */,
2DAF06D718BD1A470081CEB1 /* SmartMagnificationController.mm in Sources */,
2DE6943D18BD2A68005C15E5 /* SmartMagnificationControllerMessageReceiver.cpp in Sources */,
5272B28A1406985D0096A5D0 /* StatisticsData.cpp in Sources */,
