[kdeconnect] [Bug 370919] Keyboard to smartphone

[Albert Vaca]

https://bugs.kde.org/show_bug.cgi?id=370919

Albert Vaca  changed:

   What|Removed |Added

 CC||ci3...@gmail.com

--- Comment #16 from Albert Vaca  ---
*** Bug 367877 has been marked as a duplicate of this bug. ***

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdeconnect] [Bug 370919] Keyboard to smartphone

[Albert Vaca]

https://bugs.kde.org/show_bug.cgi?id=370919

Albert Vaca  changed:

   What|Removed |Added

 Resolution|--- |FIXED
 Status|UNCONFIRMED |RESOLVED

--- Comment #15 from Albert Vaca  ---
It has been implemented, will be part of the next release.

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdeconnect] [Bug 370919] Keyboard to smartphone

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=370919

azr...@googlemail.com changed:

   What|Removed |Added

 CC||azr...@googlemail.com

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdeconnect] [Bug 370919] Keyboard to smartphone

[Christoph Feck]

https://bugs.kde.org/show_bug.cgi?id=370919

Christoph Feck  changed:

   What|Removed |Added

 Status|RESOLVED|UNCONFIRMED
 Resolution|FIXED   |---

--- Comment #14 from Christoph Feck  ---
Comment from comment #13 was reverted, reopening.

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdeconnect] [Bug 370919] Keyboard to smartphone

[Aleix Pol]

https://bugs.kde.org/show_bug.cgi?id=370919

Aleix Pol  changed:

   What|Removed |Added

 Resolution|--- |FIXED
  Latest Commit||https://commits.kde.org/kde
   ||connect-kde/040ad7357b5a630
   ||f2ace6a29d1be5103c663
 Status|UNCONFIRMED |RESOLVED

--- Comment #13 from Aleix Pol  ---
Git commit 040ad7357b5a630f2ace6a29d1be5103c663 by Aleix Pol, on behalf of
Holger Kaelberer.
Committed on 10/01/2017 at 20:12.
Pushed by apol into branch 'master'.

kdeconnect-kde: Add remotekeyboard plugin

Allow to inject keypress events to remote peers (most notably Android devices)

Notes / open issues / possible improvements:

- For the json-payload I used the syntax of the key-events as sent by
mousepad-plugin with the addition of a "sendAck"-flag. If "sendAck" is set to
true the remote peer should echo a key-event if it could be handled, thus
allowing the local client to find out whether the key was accepted. For
performance reasons, it's allowed to send multi-char strings in the "key"
property (performs much better if you send a whole string via "echo '...' | 
kdeconnect-cli ..." e.g.)

- kdeconnect-cli: For now takes a string and transforms it into single
key-events for visible characters only. In a first implementation I used a
kbhit() helper that used termios.h to catch and relay keypresses interactively
(including some special-events), which was not optimal. A better approch might
be to use linux input-api directly. Would this be an option regarding
cross-platform compatibility or can I assume to develop for Linux only? Being a
command-line guy, I'd really like to have a fully featured kdeconnect-cli
interface ;-)

- Factor out the Qt::Key-to-internal keymap to some core-helper because it
corresponds to the mapping in the mousepad-plugin?

- The plasmoid is not perfect as it is: A single line containing a non-echoing
TextField (i.e. it eats all the KeyPress events), and only ack-ed
keypress-packets from the peer device are injected if they contain visible
keys. Advantage: the user sees whether his key-presses are accepted by the peer
device. Disadvantage: The echoed text does not correspond 1:1 to what is shown
on the peer's display, user might be confused when typing without success. I
played around with different variations each of which with its proper
shortcomings:
1. An echoing Textfield for typing: Has the advantage that the user can
directly see what he is typing, which makes interaction in the typing field
easier, BUT messes up interaction if the Editor on the peer is changed silently
and does not notify the user if his keypresses are not handled by the peer.
2. A non-echoing TextField for typing PLUS a readonly one for printing visible
echoed keys. Disadvantage: same as for the previous one and uses more space on
the plasmoid.
Comments? Ideas?

REVIEW: 129727

M  +20   -0cli/kdeconnect-cli.cpp
M  +1-0interfaces/CMakeLists.txt
M  +7-0interfaces/dbusinterfaces.cpp
M  +10   -0interfaces/dbusinterfaces.h
M  +9-0plasmoid/declarativeplugin/kdeconnectdeclarativeplugin.cpp
M  +60   -0plasmoid/package/contents/ui/DeviceDelegate.qml
A  +71   -0plasmoid/package/contents/ui/RemoteKeyboard.qml [License:
GPL (v2/3)]
M  +1-0plugins/CMakeLists.txt
M  +1-1plugins/mousepad/kdeconnect_mousepad.json
A  +8-0plugins/remotekeyboard/CMakeLists.txt
A  +23   -0plugins/remotekeyboard/README
A  +28   -0plugins/remotekeyboard/kdeconnect_remotekeyboard.json
A  +144  -0plugins/remotekeyboard/remotekeyboardplugin.cpp [License:
GPL (v2/3)]
A  +65   -0plugins/remotekeyboard/remotekeyboardplugin.h [License: GPL
(v2/3)]

https://commits.kde.org/kdeconnect-kde/040ad7357b5a630f2ace6a29d1be5103c663

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdeconnect] [Bug 370919] Keyboard to smartphone

[Simon Andric]

https://bugs.kde.org/show_bug.cgi?id=370919

Simon Andric  changed:

   What|Removed |Added

 CC||simonandr...@gmail.com

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdeconnect] [Bug 370919] Keyboard to smartphone

[Holger Kaelberer]

https://bugs.kde.org/show_bug.cgi?id=370919

--- Comment #12 from Holger Kaelberer  ---
(In reply to OlafLostViking from comment #10)
> For a functionality that lets me control the phone on my desktop (the best
> would be just "mirror" one Android app into a window on my desktop) another
> bug report would be better, right? Or would that fit into here, too?

Sounds like a new (and different) feature, like a vnc connection or so. ;-) Not
sure what exists in the Android world for such use-cases.

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdeconnect] [Bug 370919] Keyboard to smartphone

[Holger Kaelberer]

https://bugs.kde.org/show_bug.cgi?id=370919

--- Comment #11 from Holger Kaelberer  ---
kde: https://git.reviewboard.kde.org/r/129727/
Android: https://git.reviewboard.kde.org/r/129728/

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdeconnect] [Bug 370919] Keyboard to smartphone

[OlafLostViking]

https://bugs.kde.org/show_bug.cgi?id=370919

--- Comment #10 from OlafLostViking  ---
I'm happy to read about your plans - thank you!

For a functionality that lets me control the phone on my desktop (the best
would be just "mirror" one Android app into a window on my desktop) another bug
report would be better, right? Or would that fit into here, too?

(Background: using Android applications like single-device Telegram with
encrypted chats on the desktop to not always switch between devices)

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdeconnect] [Bug 370919] Keyboard to smartphone

[OlafLostViking]

https://bugs.kde.org/show_bug.cgi?id=370919

OlafLostViking  changed:

   What|Removed |Added

 CC||olaf.the.lost.viking@gmail.
   ||com

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdeconnect] [Bug 370919] Keyboard to smartphone

[Albert Vaca]

https://bugs.kde.org/show_bug.cgi?id=370919

--- Comment #9 from Albert Vaca  ---
Awesome :)

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdeconnect] [Bug 370919] Keyboard to smartphone

[Albert Vaca]

https://bugs.kde.org/show_bug.cgi?id=370919

--- Comment #7 from Albert Vaca  ---
mobile-to-desktop and desktop-to-mobile should be in different plugins (like in
the case of notifications), because they are not "symmetrical": a device can
support only receiving keys but not sending keys, so we want to enable/disable
them separately. Sorry if that was not clear in my previous message.

Ideally, as you said, keyboard and mouse should also be different plugins, but
don't worry about splitting them now if you don't want: On the Android side
they are quite coupled so it might be a bit involved to separate them.

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdeconnect] [Bug 370919] Keyboard to smartphone

[Holger Kaelberer]

https://bugs.kde.org/show_bug.cgi?id=370919

--- Comment #6 from Holger Kaelberer  ---
Thanks for your feedback.

(In reply to Albert Vaca from comment #3)
> I think this is a needed addition to kdeconnect. About the format of the
> packets, ideally it should be the same as the one used for desktop to phone
> communication. This way, it will work desktop to desktop (eg:sending keys to
> a Raspberry from my main desktop) and phone to phone too. Splitting it in
> separate plugins makes sense

Ok, then I will

1. split the mousepad-plugin into mousepad and remotekeyboard (new;
mobile-to-desktop) and

2. add the desktop-to-mobile remote-keyboard part to the (newly created)
remotekeyboard plugin.

Right?

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdeconnect] [Bug 370919] Keyboard to smartphone

[Albert Vaca]

https://bugs.kde.org/show_bug.cgi?id=370919

--- Comment #5 from Albert Vaca  ---
Oh, and about the UI... I'm not sure what's the best way to present it to the
user on the desktop, but the text field in the plasmoid probably works. About
the phone side, registering it as a keyboard is the only choice available that
I'm aware of. I'm not sure if it's even possible to make it work when the
keyboard is not shown, but that would be even better.

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdeconnect] [Bug 370919] Keyboard to smartphone

[Albert Vaca]

https://bugs.kde.org/show_bug.cgi?id=370919

--- Comment #4 from Albert Vaca  ---
[continues previous comment] makes sense for capabilities to be able to disable
the part that is actually not available without disabling other features that
do work (that's what happens if we put more than one feature in the same
plugin). 

Sent from a phone, excuse typos!

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdeconnect] [Bug 370919] Keyboard to smartphone

[Albert Vaca]

https://bugs.kde.org/show_bug.cgi?id=370919

--- Comment #3 from Albert Vaca  ---
I think this is a needed addition to kdeconnect. About the format of the
packets, ideally it should be the same as the one used for desktop to phone
communication. This way, it will work desktop to desktop (eg:sending keys to a
Raspberry from my main desktop) and phone to phone too. Splitting it in
separate plugins makes sense from

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdeconnect] [Bug 370919] Keyboard to smartphone

[Holger Kaelberer]

https://bugs.kde.org/show_bug.cgi?id=370919

--- Comment #2 from Holger Kaelberer  ---

(In reply to Holger Kaelberer from comment #1)

> * On-the-wire format:

Oups, I just realized that I forgot doing the most obvious thing: look at how
keys are serialized in the mousepad plugin for mobile-to-desktop key events. I
did now and found, that what you used could also serve my purposes, if I map
the QKeyboardEvent.key to the specialKey property as used in a
PACKAGE_TYPE_MOUSEPAD_REQUEST.

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdeconnect] [Bug 370919] Keyboard to smartphone

[Holger Kaelberer]

https://bugs.kde.org/show_bug.cgi?id=370919

--- Comment #1 from Holger Kaelberer  ---
I've got an experimental implementation of an InputMethod on Android to inject
key-events from desktop-to-mobile. After some testing and improvements it seems
to work pretty well for printable keys and some selected special keys (like
Backspace/Arrow-keys/Return etc) and I wanted to ask your opinion on some
points, Aleix and Albert (especially because I'm not an Android programmer --
yet ;-):

Some misc. remarks/questions:

* InputMethod and activation:

For now the InputMethodService is baked into the MousePadPlugin. As soon as
installed it is visible and selectable from the IME list in the Android
settings.

Once selected it remains the active InputMethod until it is deselected again by
the user or kdeconnect is uninstalled. That means, it remains active also if
not currently paired (with a remote device that supports the plugin).

For me this behaviour was fine, but I'm not sure if it should remain like that.
Depending on what is possible on android we could consider to ...

... De/Activate InputMethod automatically if not paired (if possible)?
... De/activate it based on capabilities of the paired device

For now keys are only accepted when the keyboard is visible
(InputMethodService.onStartInputView()). I think about adding a setting to
enable accepting keys always (e.g. for selecting and starting apps on the home
screen with arrow-/return-keys.

On the visual side the keyboard only contains one row with a settings- and a
keyboard-button for entering directly the settings dialog and the keyboard
selection-dialog, resp. I think about adding a isPaired indicator and maybe a
Return and a Backspace key.

* On-the-wire format:

I experimented a bit with the structure of the NetworkPackage payload (type
"kdeconnect.keyboard"). Mere transferring of printable characters turned out to
be insufficient, because it would become hard to handle special keys like
backspace, Shift, etc. Therefore for now the payload basically contains a
QKeyEvent serialized to JSON. This carries all information necessary to handle
special and modifier keys. The Qt keycodes are then mapped to
KeyEvent.KEYCODE_XXX values and some selected keys are handled in a special way
(like Backspace, Left, Shift-Left, Home, etc.) and result in KeyEvent sequences
on the android side.

One could also think about using a (single-)char utf8 payload for the
keyevents, mapping (selected) special keys to (corresponding) unicode-values.
But this makes serialization on desktop-side more complex, and I'd prefer to
stick to the QKeyEvent way.

If a keyevent was successfully delivered to the InputMethod it is reported back
to the remote keyboard (if requested), to allow for visualizing what arrived on
the other side.

It might be useful to add additional packages to signal keyboard-activation and
disable typing on a remote keyboard if it is not possible to deliver anything
or/and error-reporting to notify a remote typer why his keys did not reach
their destination.

* Plasmoid:

To the plasmoid I added a TextField for inputting the remote keys and a second
readonly one that echoes the keys confirmed from the remote side. I'm not yet
completely happy with that. Ideally I think of a single TextField (or
TextInput) not echoing keypresses immediately but only after having been ack-ed
by the remote device, thus the user can see immediately what he typed remotely.
But that would mean to interpret the echoed kepress-event equivalently to what
is done on the remote side. I tried to manually inject a QKeyPress via
postEvent directly to the TextField but that was not evaluated by the
TextField, dunno why.

Also added a command-line flag to kdeconnect-cli for sending keypresses
(interactively) to a remote.

* Plugin:

For my POC everything lives inside the MousepadPlugin on both sides. As you
already factored out the sendnotifications I wonder if I should add a new
plugin for this direction of the remote-keyboard feature as well?

* More possibilities: Add support for defining special keys on the smartphone
(e.g. F1-F10) and make them execute special actions when pressed on the remote
keyboard (Press menu button, start app X, ...)

* Finally: Would you be interested in accepting it in KDE Connect?

Please let me know what you think about it. After these first tests it looks
promising and might be a nice extension especially for computer people that
hate typing on an on-screen keyboard, like me :-P

Thanks, regards,
  Holger

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdeconnect] [Bug 370919] Keyboard to smartphone

[Holger Kaelberer]

https://bugs.kde.org/show_bug.cgi?id=370919

Holger Kaelberer  changed:

   What|Removed |Added

 CC||aleix...@gmail.com,
   ||holge...@elberer.de

-- 
You are receiving this mail because:
You are watching all bug changes.