Методы интеллектуализации управления развитием персонала высокотехнологичных сервис-ориентированных компаний тема диссертации и автореферата по ВАК РФ 00.00.00, кандидат наук Береснев Артем Дмитриевич

• Береснев Артем Дмитриевич
• кандидат наук
• 2024

Оглавление диссертации кандидат наук Береснев Артем Дмитриевич

Реферат

Synopsis

Введение

ГЛАВА 1 Проблемно-ориентированный анализ процесса управления развитием персонала высокотехнологических сервис-ориентированных компаний

1.1 Роль персонала в деятельности высокотехнологических сервис-ориентированных компаний

1.2 Анализ процесса развития персонала высокотехнологических сервис-ориентированных компаний

1.3 Анализ проблематики управления процессом развития персонала высокотехнологических сервис-ориентированных компаний

1.4 Постановка цели и задач исследования

Выводы по главе

ГЛАВА 2 Построение параметрической модели для ранжирования форм организации обучения персонала высокотехнологических сервис-ориентированных компаний на рабочем месте

2.1 Анализ существующих моделей обучения в контексте описания процесса развития персонала

2.2 Построение параметрической модели обучения персонала на рабочем месте

2.3 Ранжирование форм организации обучения персонала на рабочем месте

2.4 Метод проблемно-ориентированного ранжирования форм организации обучения персонала на рабочем месте

Выводы по главе

ГЛАВА 3 Разработка комплекса методов управления развитием персонала высокотехнологичных сервис-ориентированных компаний на основе машинного обучения

3.1 Разработка метода развития персонала через параметрическое управление поведением при Web -поиске

3.2 Разработка метода развития персонала через эмуляцию работы в продуктовой среде

3.3 Разработка метода развития персонала через обмен корпоративными знаниями

3.3.1 Анализ работы службы технической поддержки ВСОК как информационно-образовательной среды

3.3.2 Исследование возможностей интеллектуальной вопросно-ответной системы на базе большой языковой модели для развития персонала технической поддержки

3.3.3 Разработка и исследование интеллектуальной вопросно-ответной системы на основе эксплицитного представления знаний для развития персонала технической поддержки

3.3.4 Разработка и исследование интегрированной интеллектуальной вопросно-ответной системы для развития персонала технической поддержки

3.3.5 Метод развития персонала технической поддержки путем обмена корпоративными знаниями

Выводы по главе

ГЛАВА 4 Разработка интегрированного программного комплекса, реализующего методы управления развитием персонала высокотехнологичных сервис-ориентированных компаний

4.1 Разработка программного решения для развития персонала через параметрическое управление поведением при Web -поиске

4.2 Разработка программного решения для развития персонала через эмуляцию работы в продуктовой среде

4.3 Разработка программного решения для развития персонала через обмен корпоративными знаниями

Выводы по главе

ГЛАВА 5 Экспериментальная оценка эффективности применения разработанного комплекса методов интеллектуализации управления развитием персонала высокотехнологичных сервис-ориентированных компаний

5.1 Анализ структуры и деятельности типичной высокотехнологичной сервис-ориентированной компании

5.2 Организация эксперимента

5.3 Результаты и обсуждение

5.3.1 Оценка повышения эффективности организации технологического процесса предоставления сервиса

5.3.2 Оценка повышения эффективности работы персонала

5.3.3 Анализ результатов использования методов управления развитием

персонала

Выводы по главе

Заключение

Список литературы

Приложение 1 Оценка значимости атрибутов параметрической таблицы применимости форм организации обучения на рабочем для

высококвалифицированных сотрудников

Приложение 2 Пример письма с рекомендациями по коррекции поискового поведения

Приложение 3 Пример описания конфигурации сети в формате XML

Приложение 4 Документы об использовании и внедрении результатов диссертационного исследования

Приложение 5 Публикации автора по теме диссертации

Реферат

Введение диссертации (часть автореферата) на тему «Методы интеллектуализации управления развитием персонала высокотехнологичных сервис-ориентированных компаний»

Общая характеристика диссертации Актуальность темы исследования. Современные

высокотехнологичные компании, особенно в 1Т-секторе, чаще всего используют сервисную бизнес-модель, которая основана на предоставлении заказчику постоянного 1Т-сервиса или услуги - значимого для бизнеса результата, вместо однократной покупки продукта как средства получения результата. Такой подход позволяет заказчикам оптимизировать затраты за счет делегирования производственных функций сервисным организациям.

Актуальность работы обусловлена увеличением доли высокотехнологических компаний с сервисной моделью ведения бизнеса, в том числе и среди компаний малого и среднего размера [96, 93]. Ключевым ресурсом, влияющим на конечный успех деятельности таких высокотехнологичных сервис-ориентированных компаний (далее ВСОК), становится персонал, обладающий техническими знаниями и навыками, инновационной активностью, креативностью, ответственностью за результат. Это подтверждается тем, что в актуальных отраслевых методиках управления ГГ-сервисами (ГПЬ [61], CoBIT [40] и др.) персонал и его качество рассматривается как один из ключевых ресурсов. Для достижения результата в процессе работы персоналу ВСОК необходимо повышать уровень технических знаний и навыков, осваивать и применять новые инновационные технологические решения и управленческие практики.

В современной практике управления персоналом большое внимание уделяется автоматизации и интеллектуализации внутренних бизнес-процессов, проектного управления, юридического сопровождения, опросов и т. п. Однако в общей структуре процесса управления персоналом имеются аспекты, интеллектуализации которых до сих пор уделялось недостаточно внимания, среди них - управление развитием сотрудников современных ВСОК и его инструментальная поддержка.

Существующие исследования, рассматривающие средства и методы управления развитием сотрудников, в основном адресуются к персоналу начального уровня, в то время как для высококвалифицированного персонала эти вопросы проработаны мало. С другой стороны, крупные 1Т-компании, ввиду практического отсутствия финансовых ограничений, могут привлекать необходимое число специалистов нужного уровня со стороны, в то время как для малых и средних ВСОК проблема целенаправленного развития высококвалифицированного персонала является одной из определяющих в успехе бизнеса. Причем, как показывает анализ литературы, такое развитие реализуется через обучение на рабочем месте.

Таким образом, имеет место противоречие между необходимостью развития высококвалифицированного персонала высокотехнологических сервис-ориентированных компаний малого и среднего размера.

Степень проработанности темы. Описанные выше понятия и проблемы нашли свое отражение в работах отечественных и зарубежных авторов. Понятие ВСОК рассмотрено в работах [96, 93, 20, 14, 81] и др. Исследователи выделяют ключевые особенности деятельности ВСОК, такие как высокая динамика процессов, важность коммуникации с клиентом, необходимость быстрого внедрения новых технологий, описывают уровни зрелости сервисов, состав и специфику управления ресурсами предприятия.

Характеристики персонала ВСОК рассмотрены в работах [8, 11, 47, 60, 81 ] и др. Исследователи выделяют как исключительно значимые способность сотрудников компании решать слабо формализованные задачи и, в связи с этим, особую роль высококвалифицированного персонала.

Вопросы развития персонала ВСОК освещены в работах [8, 10, 22, 24]. Авторы рассматривают сущность и подходы к организации развития персонала как комплексного процесса. Так, например, рассматривается статистический, динамический, функциональный, процессный, ресурсный и интегративно-конвергенциальный подход к формированию системы управления развитием персонала высокотехнологичных компаний.

Обосновываются преимущества для управления развитием персонала непрерывного использования анализа текущего профессионального опыта сотрудников. Исследователи отмечают, что в контексте ВСОК развитие персонала тесно связано, и даже определяется, его обучением.

Содержательно методы и подходы к обучению персонала рассмотрены в работах [9, 12, 13, 25, 41, 43, 58] и др. В своих работах авторы приводят классификации методов и средств обучения персонала, описывают их ограничения. Признается тот факт, что значительный объем необходимых для работы знаний персонал ВСОК получает в процессе так называемого неформального обучения на рабочем месте [1, 15, 26, 38, 43, 50, 68, 72, 87].

Обучению на рабочем месте посвящены работы [1, 15, 25, 26, 55, 58, 63, 66, ]. Авторы описывают модели обучения на рабочем месте, раскрывают содержание процесса, существующие практики и ограничения обучения на рабочем месте.

Специфика управления обучением на рабочем месте в контексте развития персонала ВСОК и интеллектуализация управления этого процесса рассмотрены в работе [5], в которых описываются процессные подходы к построению многомерных моделей развития, позволяющие определять параметры процесса управления. Исследователи [7] рассматривают теоретико-игровые и оптимизационные модели управления развитием персонала. Вводится система классификаций задач управления развитием персонала, проводится краткий обзор основных подходов к этой проблеме с точки зрения различных наук. Рассматриваются модели индивидуального развития: иерархии потребностей, управления профессиональной адаптацией, мотивации, управления карьерой.

Вопросы разработки специализированных АСУ для управления развитием персонала рассматриваются в работе [34]. Авторы предлагают подход, основанный на многоцелевой модели принятия решений, учитывающей множество критериев, включая профиль учащегося, стиль обучения, уровень компетентности, доступное время. Описанный

программный комплекс предполагает предварительную подготовку учебных материалов с использованием онтологий для моделирования соответствующих доменов и ведения профилей учащихся. В работе [50] авторы предполагают использование стандартных систем управления обучением (англ. Learning Management System, LMS) общего назначения или даже обычных инфо-коммуникативных средств для организации управления обучения на рабочем месте, предполагая, что вопросы организации, отбора учебных материалов, обратной связи с персоналом и интеграции обучения в повседневную рабочую деятельность будут решаться административно руководством с привлечением экспертов.

Широко известны стратегии управления персоналом в высокотехнологических компаниях, а именно:

1) стратегия динамического роста, направленная на повышение уровня сотрудничества внутри команды и способности сотрудников быстро адаптироваться к любым переменам,

2) стратегия предприимчивости, направленная на развитие новаторства, инициативности и высокой ответственности за результат,

3) стратегия слаженности и контроля во имя прибыли, при которой фокус делается на соблюдении сроков выполнения задач и достижения запланированных результатов,

4) ликвидационная стратегия и стратегия сокращения, реализуемые в неблагоприятных для бизнеса экономических условиях, но требующих от сотрудников расширения круга профессиональных задач и быстрого освоения новых технологий.

Анализ содержания стратегий доказывает исключительную важность обучения персонала высокотехнологических компаний.

Анализ приведенных исследований позволяет утверждать, что для ВСОК ввиду специфики их деятельности и особых требований к развитию персонала, и особенно для персонала высокой квалификации, обучение на рабочем месте наилучшим образом соответствует условиям и целям

развития. Существуют модели обучения, позволяющие описывать процесс и условия обучения на рабочем месте, однако они носят общий характер, не являются исчерпывающими по отношению к развитию именно персонала ВСОК и не несут в себе специфичных для ВСОК параметров управления. Имеющиеся процессные модели могут применяться для административного управления и целеполагания, однако при этом вопросы алгоритмического, методического и программного обеспечения развития высококвалифицированного персонала ВСОК пока практически не проработаны. Таким образом, анализ существующих исследований по теме диссертации подтверждает ее актуальность.

Объект исследования: процесс развития персонала высокотехнологических сервис-ориентированных компаний малого и среднего бизнеса.

Предмет исследования: интеллектуальная поддержка управления развитием персонала высокотехнологических сервис-ориентированных компаний малого и среднего бизнеса.

Целью исследования является повышение эффективности персонала высокотехнологичных сервис-ориентированных компаний малого и среднего бизнеса посредством интеллектуализации управления его развитием.

Для достижения цели в ходе исследования ставятся следующие задачи:

1) Разработка метода параметрического ранжирования форм организации обучения персонала ВСОК для выделения наиболее подходящих методов обучения на рабочем месте.

2) Разработка метода управления развитием персонала через непрямое управление поведением сотрудников при Web-поиске для совершенствования у персонала навыков поиска и более эффективного использования рабочего времени.

3) Разработка метода управления развитием персонала через эмуляцию работы в продуктовой среде с целью сокращения времени

освоения сотрудником новых технологий и увеличения качества предоставляемого клиентам сервиса со стороны малых и средних ВСОК.

4) Разработка метода управления развитием персонала через обмен корпоративными знаниями для предоставления сотруднику информации, релевантной контексту производственной задачи, при одновременном сокращении нагрузки на наставника.

5) Разработка интегрированного программного комплекса управления развитием персонала ВСОК, реализующего разработанные методы, и экспериментальная оценка их эффективности.

Методы и средства исследования. При решении задач исследования применялись элементы теории грубых множеств, методы машинного обучения (тематического моделирования), анализа формальных концептов, наблюдения и эксперимента.

Положения, выносимые на защиту, обладающие научной новизной. Получены следующие результаты, выносимые на защиту:

1) Метод параметрического ранжирования форм организации обучения персонала ВСОК, отличающийся тем, что, с целью формализованного выявления наиболее подходящих для конкретных условий форм организации обучения персонала ВСОК, разработана параметрическая модель обучения на рабочем месте с расширенным набором параметров, специфичных для ВСОК

2) Метод управления развитием высококвалифицированного персонала, реализующий разработанную модель и основанный на эмуляции продуктовой среды заказчика на стороне ВСОК, отличающийся тем, что, с целью сокращения разрыва между средой проектирования и продуктовой средой, применяется новая архитектура эмулируемой среды и автоматизированная проверка ее конфигурации.

3) Методы управления развитием высококвалифицированного сотрудника, отличающиеся тем, что, с целью повышения степени использования результатов текущей профессиональной деятельности в

процессе его развития применяется непрямое управление его поисковым поведением и средство интеллектуальной поддержки обмена корпоративными знаниями.

Степень достоверности результатов. Достоверность научных положений и выводов, полученных в диссертационной работе, подтверждается экспериментальными исследованиями, их соответствием результатам, полученным ранее в данной предметной области, успешным представлением основных положений в докладах на международных и российских конференциях, а также публикациями в рецензируемых изданиях, положительным опытом внедрения.

Соответствие паспорту специальности. В соответствии с паспортом специальности 2.3.4. «Управление в организационных системах» диссертация относится к областям исследований: 5. Разработка методов получения данных и идентификации моделей, прогнозирования и управления организационными системами на основе ретроспективной, текущей и экспертной информации; 9. Разработка методов и алгоритмов интеллектуальной поддержки принятия управленческих решений в организационных системах.

Теоретическое значение работы состоит в развитии теории управления организационными системами в контексте методов и алгоритмов интеллектуальной поддержки принятия управленческих за счет разработки метода, позволяющего расширить возможности ранжирования компонентов универсума в теории грубых множеств на элементы, описываемые разнотипными признаками, посредством модификации и применения метрики Гувера.

Практическое значение работы состоит в том, что на основе разработанных методов создан интегрированный программный комплекс интеллектуализации управления развитием персонала, который может быть использован и частично используется в настоящее время для решения задач оптимизации бизнес-процессов в аспекте управления человеческими

ресурсами. В состав комплекса входят средство параметрического управления поведением сотрудника при Web-поиске, реализованного в виде программного решения, анализирующего текущий поисковую историю сотрудника в Интернет и позволяющего сократить пространство поиска и оптимизировать использование рабочего времени, программное решение для развития персонала через эмуляцию работы в продуктовой среде, позволяющее персоналу быстрее осваивать сетевые технологии и программные средства и эффективнее предоставлять услуги заказчикам, программное решения для развития персонала через обмен корпоративными знаниями в виде интегрированного чат-бота, предоставляющего сотруднику оперативно получать доступ к рекомендованной информации, релевантной контексту производственной задачи и снижать нагрузку на экспертов. Применение комплекса в компании ООО «РОТЭК» привело к повышению эффективности работы сервис-инженеров по ключевым показателям от 31 до 54 %, что подтверждено актом внедрения. Результаты диссертационной работы были использованы при выполнении НИР № 390333 «Разработка алгоритмов распознавания медико-биологических объектов методом переноса знаний с использованием глубоких нейронных сетей».

Основные результаты работы докладывались и обсуждались на следующих конференциях: «Digital Transformation and Global Society -2020» (Russia, Saint-Petersburg), «2nd International Conference on Geoinformatics and Data Analysis (ICGDA2019)» (Prague, Czech Republic), «The 12th annual International Conference of Education, Research and Innovation ICERI 2019» (Spain, Valencia), II Международная научная конференция «Технологическая перспектива в рамках Евразийского пространства: новые рынки и точки экономического роста - 2016» (Россия, Санкт-Петербург), «LIII научная и учебно-методическая конференция Университета ИТМО 2024» (Россия, Санкт-Петербург), «VIII Конгресс молодых ученых - 2019» (Россия, Санкт-Петербург).

Публикации. По результатам исследования опубликовано семь публикаций, из них пять опубликованы в изданиях, индексируемых в базе цитирования Scopus, три - Web of Science и одна - РИНЦ.

Личный вклад автора. Автором лично проведен анализ существующих подходов к управлению развитием персонала [33], на основании которого построена проблемно-ориентированная модель для ранжирования форм организации развития персонала ВСОК [31]. Лично автором разработаны метод развития персонала через управление поведением сотрудников при Web-поиске [30], метод управления развития персонала через эмуляцию работы в продуктовой среде [29], метод управления развития персонала через обмен корпоративными знаниями [32, 33]. Лично автором разработан интегрированный программный комплекс

интеллектуализированного управления развитием персонала ВСОК [3] и проведена экспериментальная оценка его эффективности [2]. Две публикации [2, 3] подготовлены без соавторства, подготовка пяти публикаций проводилась с соавторами, при этом вклад автора был основным в публикациях с первым авторством.

С соавтором публикации [31] Гусаровой Н.Ф. проводились обсуждения возможных вариантов моделирования. Соавторы публикаций [32, 33] Жданкин А.М., Лобанцев А.А., Бойцов В.А., Егоров М.В. выполняли расчеты^ связанные с реализацией метода формального анализа компонентов. Соавторы работы [29] Артемова Г.О., Бурая К.И. программно реализовали эмулирующую среду. В работе [30] соавторы Добренко Н.В. и Ватьян А.С. выполнили подбор и подготовку датасетов, Дудоров С.В. программно реализовал отдельные методы тематического моделирования.

Внедрение результатов работы. По отдельным результатам диссертационной работы были получены зарегистрированные результаты интеллектуальной деятельности:

Гусарова Н.Ф., Шалыто А.А., Береснев А.Д., Добренко Н.В., Ватьян А.С., Жданкин А.М. «Утилита для визуализации концептуальной структуры

домена» // Свидетельство о государственной регистрации программы для ЭВМ № 2019613715. Дата регистрации - 21.03.2019.

Гусарова Н.Ф., Береснев А.Д., Мамедов Н.З. «Модульный чат-бот для тестирования» // Свидетельство о государственной регистрации программы для ЭВМ № 2018612516. Дата регистрации от 19.02.2018.

Разработанный программный комплекс был внедрен в компании ООО «РОТЭК» и использовался для развития инженерного персонала компании. Успешность внедрения подтверждается соответствующим актом.

Структура и объем диссертации. Диссертация состоит из введения, пяти глав и заключения. Полный объем диссертации составляет 343 страниц текста с 35 рисунками и 27 таблицами. Список литературы содержит 96 наименование.

Публикации автора по теме диссертации

Публикации в периодических изданиях, входящих в перечень Web of Science и Scopus:

Artemova G.O., Beresnev A.D., Buraya K.I., Gusarova N.F. System of network infrastructure configuring for Project Learning // Journal of Engineering and Applied Sciences, 2017. 12 (special issue 9), pp. 8607-8613.

Beresnev A., Dudorov S., Gusarova N., Dobrenko N., Vatyan A., Nigmatullin N., Vedernikov N., Vasilev A., Shalyto A. Topic modeling of text content for monitoring the employee's efficiency via his Internet activity / Multi Conference on Computer Science and Information Systems; Proceedings of the International Conferences on Big Data Analytics, Data Mining and Computational Intelligence 2018, Theory and Practice in Modern Computing 2018 and Connected Smart Cities 2018, MCCSIS 2018 - 2018, pp. 43-50.

Beresnev A.D., Boytsov V.V., Dobrenko D.A., Egorov N.V., Gusarova N.F. Workplace learning for personnel of high-tech service-oriented companies / 12th annual International Conference of Education, Research and Innovation (ICERI 2019 Proceedings), pp. 361-372, 2019. doi: 10.21125/iceri.2019.2363.

Береснев А.Д. Интеллектуализация управления развитием персонала высокотехнологичных сервис-ориентированных компаний // Научно-технический вестник информационных технологий, механики и оптики [Scientific and Technical Journal of Information Technologies, Mechanics and Optics]. 2022. Т. 22. № 1 (137). С. 127-137.

Beresnev A., Gusarova N. Comparison of Intelligent Classification Algorithms for Workplace Learning System in High-Tech Service-Oriented Companies // Communications in Computer and Information Science. 2020, Vol. 1242, pp. 363-372.

Beresnev A., Zhdankin A., Lobantsev A., Vasiliev A., Vedernikov N., Gusarova N. Dialogue System for Service Desk of Complex Software Systems Based on Relational Concept Analysis // ACM International Conference, 2nd International Conference on Geoinformatics and Data Analysis - 2019, Vol. F148261, pp. 31-36.

Публикации в других изданиях:

Береснев А.Д. Применение открытых LLM с RAG-архитектурой для организации технической поддержки ИТ-сервисов / Труды Пятьдесят третьей (LIII) научной и учебно-методической конференции Университета ИТМО, 2024, с. 21-28.

Содержание работы

Во введении дана общая характеристика работы, обоснована её актуальность, сформулированы цель и задачи, научная новизна, теоретическая значимость и практическая ценность полученных результатов, приведены основные положения, выносимые на защиту.

Первая глава диссертации посвящена проблемно-ориентированному анализу процесса управления развитием персонала высокотехнологических сервис-ориентированных компаний.

В разделе 1.1 рассмотрены вопросы деятельности персонала высокотехнологических сервис-ориентированных компаний. Определено понятие ВСОК как компании, предлагающей специализированные

высокотехнологические услуги и решения, основанные на процессах, а не только на физических продуктах. Разработан обобщенный процесс предоставления сервиса ВСОК, выявлены подпроцессы, где деятельность сотрудников наименее формализована, сформулированы требования к персоналу ВСОК, реализующему эти процессы (высокая квалификация, технический кругозор, опыт эксплуатации конкретных сервисов, умение быстро осваивать новые технологии, а также «мягкие» навыки общения и коммуникации), определена специфика деятельности персонала разных квалификационных уровней.

В разделе 1.2 проведен анализ процесса развития персонала высокотехнологических сервис-ориентированных компаний. Дано определение процесса развития персонала в контексте деятельности организаций как процесса формирования и расширения у сотрудника профессиональных компетенций, заключающихся в приобретении комплекса специфических знаний, умений и навыков, с целью подготовки сотрудников к выполнению новых или повышения эффективности выполнения текущих обязанностей, стимулирования творчества и создания условий для саморазвития и карьерного роста. Сформулированы требования к процессу (обеспечение высокой скорости освоения новых компетенций, контекстная зависимость содержания процесса развития от текущих производственных задач, растянутость во времени и непрерывность). Рассмотрены существующие подходы к управлению процессом развитием персонала. Установлено, что в основе профессионального развития персонала лежит именно его обучение. Рассмотрены формы организации обучения на рабочем месте, относящиеся к формальному и неформальному обучению. Установлено, что некоторые из них, относящиеся в основном к формальному обучению, имеют хорошо проработанную технологическую поддержку специализированными программно-аппаратными решениями, в то время как вопросы интеллектуальной инструментальной поддержки неформального обучения персонала современных ВСОК, особенно в виде самостоятельного

обучения, изучены мало. Показано, что ввиду необходимой контекстной обусловленности конкретным производственным процессом обучения персонала ВСОК и значительными различиями в мотивации и специфике деятельности персонала ВСОК разной квалификации, выбор оптимальной формы организации обучения персонала является нетривиальной задачей, которая может быть решена с помощью разработки формализованного метода, позволяющего при выборе учесть контекст задачи и ресурсные ограничения организации.

В разделе 1.3 проведен анализ проблематики управления процессом развития персонала высокотехнологических сервис-ориентированных компаний. Показано, что управление развитием персонала ВСОК как обучением на рабочем месте персонала ВСОК является сложным, многопараметрическим процессом, при этом, несмотря на большое внимание, уделенное исследователями общим вопросам управления развитием персонала ВСОК, практические вопросы реализации и алгоритмического и методического обеспечения такого управления изучены мало. Установлено, что не существует предиктивных методов выбора наиболее подходящих форм организации обучения на рабочем месте для персонала ВСОК с учетом профессионального уровня персонала и контекста производственной задачи. Обоснована целесообразность разработки формализованного метода, позволяющего при выборе формы организации обучения на рабочем месте учесть такие параметры, как контекст задачи, характеристики обучающихся и ресурсные ограничения организации, Показано, что этот список параметров следует расширить требованиями учета профессионального уровня обучаемых.

В разделе 1.4 осуществлена постановка цели исследования и сформулированы задачи исследования.

Вторая глава диссертации посвящена построению параметрической модели для ранжирования форм организации обучения персонала высокотехнологических сервис-ориентированных компаний на рабочем

месте. Разработана параметрическая модель обучения на рабочем месте, разработан универсальный метод ранжирования форм обучения на рабочем месте, выявлены наиболее подходящие формы организации обучения на рабочем месте для высококвалифицированного персонала ВСОК.

В разделе 2.1 проведен анализ существующих моделей обучения в контексте описания процесса развития персонала. Были рассмотрены обобщенная модель обучения (Галинская Е.В., Иващенко А.А., Новиков Д.А. [7]; Biggs J. & Tang C. [36]), модель педагогической системы (Симонов В.П. [21]), концептуальный фреймворк развития персонала (Jacobs R.L. [62]). Установлено, что все модели носят общий характер, не являются исчерпывающими по отношению к развитию именно персонала ВСОК малого и среднего бизнеса и не несут в себе специфичных для ВСОК параметров управления. Наиболее подходящей для решения задачи, поставленной в первой главе диссертации, оказывается параметрическая модель Симонова, при условии ее уточнения для условий обучения персонала малых и средних ВСОК.

В разделе 2.2 осуществлено построение параметрической модели обучения персонала на рабочем месте. Модель Симонова была дополнена специфичными для развития персонала ВСОК параметрами и приобрела вид кортежа (1):

PS = <T, P, C, A, F, I, R>, (1)

где PS - педагогическая система, T - цель обучения, P - участники процесса, С - содержание обучения, А - средства обучения, F - форма обучения, I -степень интегрированности в реальный рабочий процесс, R - степень использования результатов. Параметры I и R, являющиеся дополнительными, выявлены на основании анализа литературы. В работе описаны шкалы параметров уточненной параметрической модели обучения на рабочем месте. Составлена параметрическая таблица применимости форм организации обучения на рабочем для высококвалифицированных сотрудников.

Список литературы диссертационного исследования кандидат наук Береснев Артем Дмитриевич, 2024 год

Литература

1. Zakrzewska-Bielawska. High technology company - concept, nature, characteristics // Proceedings of the 8th WSEAS Int. Conf. on Management, Marketing and Finance, WSEAS Press, Penang, Malaysia, March 23. 2010. P. 93-98.

2. Wolf M., Terrell D. The high-tech industry, what is it and why it matters to our economic future // Beyond the Numbers: Employment and Unemployment. V. 5. N 8 (U.S. Bureau of Labor Statistics, May 2016).

3. Cacciattolo K. Defining workplace learning // European Scientific Journal May 2015 /SPECIAL edition. V. 1. P. 243-250.

4. Hart J. Modern Workplace Learning 2019: A Framework for Continuous Improvement, Learning and Development, https://www.modernworkplacelearning.com/cild/, last accessed 2020/02/25.

5. Sweeney F. Workplace learning: The evolution of IT learning. March 2016 [Электронный ресурс]. Режим доступа: https://www.theceomagazine.com/business/management-leadership/workplace-learning-the-evolution-of-it-learning/ , свободный. Яз. англ. (дата обращения: 20.11.2021).

6. Dreyfus Stuart E., Dreyfus Hubert L. A Five-Stage Model of the Mental Activities Involved in Directed Skill Acquisition. Washington, DC: Storming Media [Электронный ресурс]. Режим доступа: https://www.researchgate.net/publication/235125013_A_Five-Stage_Model_of_the_Mental_Activities_Involved_in_Directed_Skill_Acquisition, свободный. Яз. англ. (дата обращения: 20.11.2021).

7. Галинская Е.В., Иващенко А.А., Новиков Д.А. Модели и механизмы управления развитием персонала. М.: ИПУ РАН, 2005. 68 с.

8. Шестакова Е.В. Инновационные технологии обучения персонала. 2015 [Электронный ресурс]. Режим доступа: https://elib.osu.rU/bitstream/123456789/1230/1/2278-2283.pdf, свободный. Яз. рус. (дата обращения: 21.11.2021).

9. Кязимов К.Г. Управление человеческими ресурсами: профессиональное обучение и развитие : учебник для академического бакалавриата / К.Г. Кязимов. 2-е изд., перераб. и доп. Москва: Издательство Юрайт, 2019. 202 с.

10. Вялова Е.П. Динамическое моделирование и оптимизация производительности инновационной организации с учетом профессиональной компетенции сотрудников / Е.П. Вялова, Г.А. Квашнина, В.И. Федянин // Системы управления и информационные технологии: науч.-техн. журнал. М.: ИПУ РАН, 2006. № 4.1(26). С. 134-136.

11. Симонов В.П. Педагогический менеджмент: Ноу-хау в образовании. Москва, Высшее образование, 2006. 368 с.

12. Jacobs R.L. A proposed conceptual framework of workplace learning. Human Resource Development Review. V. 8. N 2. June 2009. P. 133-150.

13. Принятие решений в неопределенности. Правила и предубеждения / Ред. Канеман Д., Словик П., Тверски А. Гуманитарный центр, 2021. 540 с. ISBN: 978-617-7022-281

14. Трегуб И.В. Имитационные модели принятия решений. Инфра-М, 2020. 193 с.

15. Uzhga-Rebrov O. Knowledge representing features in rough sets theory // Environment. Technology. Resources Proceedings of the 7th International Scientific and Practical Conference. Volume II, Rezeknes Augstskola, Rezekne, RA Izdevnieciba, 2009.

16. Федорова Е.С., Лашманова Н.В. Применение комбинированных методов оценки в системе управления компетенциями инновационного предприятия // Дискурс. 2016. № 6. С. 56-59.

17. Hart J. Modern Workplace Learning 2020: How Modern Professionals Prefer to Learn [Online]. Retrieved from www.modernworkplacelearning.com/cild/introduction/1-7-how-modern-professionals-prefer-to-learn/

18. The Official Introduction to the ITIL Service Lifecycle. London: TSO [Электронный ресурс]. Режим доступа: http://wikiitil.ru/books/00%20ITIL3%200fficial%20Introduction.pdf свободный. Яз. англ. (дата обращения: 18.10.2021).

19. Poelmans J., Ignatov D.I., Kuznetsov S., Dedene G. Formal concept analysis in knowledge processing: A survey on applications // P. 2. Expert Systems with Applications. 2013. V. 40. N 16. P. 6538-6560.

20. Priss U. Associative and Formal Concepts // Conceptual Structures: Integration and Interfaces. Proceedings of the 10th International Conference on Conceptual Structures. LNAI 2393. 2002. P. 354-368.

WORKPLACE LEARNING FOR PERSONNEL OF HIGH-TECH SERVICE-ORIENTED COMPANIES

Beresnev1 Artem D., Boytsov1 Vitaly V., Dobrenko2 Dmitriy A., Egorov1 Nikolay V., Gusarova1 Natalia F.

11TMO University (RUSSIA) 2 Herzen State Pedagogical University (RUSSIA)

We investigate the methods and means of workplace learning for high-tech service-oriented companies' staff. We formalize the parametric model of workplace learning and approaches to its organization. With the methods of rough set theory, we determine the most appropriate approaches to the formation of specific staff competencies. Moreover, the classes of qualification of personnel who can train together are defined. We describe three specially designed means of support workplace learning, namely: the mean based on the Linux network namespace of teaching personnel in a real-work environment, the mean of answering employees' professional questions in a dialogue system based on methods of formal concept analysis, and finally, the mean for estimating employee behavior deviations from effective Internet search scenarios based on LDA and ARTM algorithms.

Encouraging results of six-month operation of the developed means and methods in one hightech service-oriented companies are given.

Keywords: workplace learning, rough set, LDA, ARTM, formal concept analysis, learning support tools. 1 INTRODUCTION

As world experience shows, business in high-tech sphere in combination with a service-oriented approach provides companies with significant competitive advantages [1, 2]. Due to the rapid progress of technology, high-tech service-oriented companies (HSOC) are forced to constantly support these advantages, and staff plays a key role here possessing key competencies, namely technical knowledge, technical skills, innovative activity, creativity, responsibility for the result. In order for employees to acquire, maintain and constantly improve these competencies, HSOC use various types of employee training, among which workplace learning plays the leading part.

Workplace learning is defined as the way in which skills are upgraded and knowledge is acquired at the place of work [3]. Methods of workplace learning are divided into formal and informal. Formal methods are well known and widely used. These include conferences, continuing education courses (including distance learning), distribution of information resources, etc. The specificity of formal methods lies in the fact that both their form and content are structured in advance and provided to students in a predefined way. Non-formal learning takes place on the personal initiative of the employee. The employee himself chooses the problems of his non-formal education; as a rule, they are determined by his private professional interests or specific professional tasks. Non-formal learning can be implemented in various forms - for example, in the form of communication with colleagues, information search in Internet, personal experiments, etc.

Modern professionals engaged in HSOC highly appreciate the effectiveness of non-formal workplace learning. For example, in the 2019 survey [4], the first places were taken by the following training methods: (1) daily work experience, (2) knowledge sharing within the work team, (3) personal and independent search on the Internet, (4) feedback from the leader / mentor. At the same time, traditional methods associated with formal training, including continuing education courses, conferences and e-learning, were in last places.

However, non-formal workplace learning also has disadvantages. Firstly, far from always the colleagues of a particular employee whom he seeks advice have sufficient experience and knowledge, as well as time to maintain a professional dialogue at the level and extent necessary for that employee. At the same time, providing each employee with the opportunity of personal communication with the facilitator (mentor) of the required level "on demand" is an unrealistic task from both an economic and organizational point of view. Secondly, informal learning at the workplace is most often carried out by the employee spontaneously, in response to problems arising from the employee, that is, is not controlled and not managed by the administration.

In connection with the growing demand for workplace learning, its various aspects are widely discussed. Most of the research touches on social, pedagogical and organizational aspects [4 - 11]. The possibilities of using the latest technologies, already familiar to the HSOC staff, like mobile applications, virtual reality tools and simulators, as training tools in the workplace learning are also being discussed last years [12]. However, the urgent question of how to turn such daily activities performed by the HSOC employees as individual web search, daily work experiences and contacts with colleagues into effective means of non-formal workplace learning for these employees at the workplace is hardly reflected in the literature.

Thus, the authors of the article set themselves the following tasks:

- to identify ways of workplace learning that best provide the needed competencies of HSOC personnel;

- to develop methods and software tools to support the identified ways of workplace learning, which reduce the need for personal contacts of the employee with the facilitator and allow indirect control of the process of non-formal learning.

2 IDENTIFICATION OF PREFERRED WAYS OF SUPPORTING WORKPLACE LEARNING FOR HSOC PERSONNEL

In order to identify ways of workplace learning that best provide the needed competencies of HSOC personnel, first of all it is necessary to establish the place of workplace learning in the overall structure of learning processes.

In accordance with the main pedagogical theories in [13] pedagogical models and frameworks are classified according to various perspectives (associative, cognitive and situative). An analysis of the literature shows that all these approaches are used to model workplace learning. For instance, in [14], a process model of non-formal and incidental learning is proposed. In [15], a model of informal learning community based on Social Networking Services is considered. The authors [16] consider the process of non-formal learning as communication within communities of practitioners, and use the characteristics of a social networks as the main parameters of the model.

Nevertheless, a meaningful analysis shows that the most promising for modeling workplace learning is the use of generic frameworks, in particular, constructive alignment [17]. The central idea of his approach is that the learning activities and assessment within a course should be aligned with the intended learning outcomes. However, a number of authors [18, 19] highlight the specifics of workplace learning in the overall structure of the learning process:

- Workplace learning combines two significant human processes (working and learning. Namely, within workplace learning, work is necessarily a part of learning and learning in turn is a part of the work.

- Workplace learning contains both formal and non-formal aspects. The first one is associated with the provision of planned training and education courses, the second - with the more informal processes that are embedded in an ad hoc personal activity.

In accordance with the variety of modeling approaches, various options for parameterizing models of the pedagogical process in general and workplace learning in particular are presented in the literature. According to [17], the model of the learning process can be represented as a tuple:

Learning process =<Learning activities, Learning outcomes, Assessment> (1)

In [20] the following model for pedagogical system (PS) as a whole is proposed:

PS = <T, P, C, A, F>, (2)

where T is the goal of PS, P is the participants of PS, C is educational content, A is learning tools; F is a form of training.

The workplace learning model proposed in [21] is based on three main parameters: location of the learning, degree of planning, role of trainer/facilitator. Each of them, according to [21], can have one of two values. The combination of possible parameter values gives eight variants that form the conceptual framework (CF) of workplace learning:

CF = < Location of the learning (Off the job / On the job); Degree of planning (Unstructured / Structured); Role of trainer/facilitator (Passive / Active)>. (3)

Each variant thus formed is matched with existing forms of workplace learning. However, as a detailed analysis shows, this matching is rather schematic, it is carried out either theoretically or with a certain degree of coincidence. For some combinations of parameters (for example, Off the job / Unstructured / Active) in reality, there are no corresponding forms of workplace learning. At the same time, there are combinations of parameters (for example, On the job / Unstructured / Passive) that simultaneously correspond to several forms of workplace learning, most of which can be applied to HSOC, and so the differences between them remains unexplained.

Thus, despite the variety of educational process models and their parameterizations proposed in the literature, not one of them is exhaustive in relation to the workplace learning of the HSOC personal. Therefore, to form a problem-oriented model of the workplace learning of the HSOC personal, we use a combined approach. Namely, since the subject of the article is personnel with a high educational level, as the basis we take [17] model (1), which is widespread in the higher education domain, and determine the semantic content of each parameter in relation to the task of workplace learning support in HSOC.

Consider the proposed approach in more detail. The "learning outcomes" parameter from (1) correlating with the parameter T from (2) may be presented as the desired competencies of the employee. Based on the literature [4, 22-24] analyzed, we have identified the list of desired competencies of HSOC staff, presented in Table 1 (upper row).

We characterize the process participants by their place in the production hierarchy, which is associated with the levels of responsibility and of tasks they can solve, which corresponds to the parameter "assessment" in the model (1) and to the parameter P in the model (2). An analysis of the literature [21] reinforced by a survey of HSOC staff shows that for the HSOC a five-level hierarchy of personnel is characteristic, which is presented in Table 1 (middle row).

Table 1. Parameters of of the developed workplace learning model.

Parameter according to (1) / (2) Designation Content

Learning outcomes / T a1 High level of specialized knowledge

a2 Work in conditions of uncertainty and constant

changes, low threshold for entering new tasks

a3 Mobility and diversity teamwork skills

a4 Creative thinking

a5 Responsibility for self-solving problems

a6 Continuing education, including on mistakes,

professional development and development

a7 The prevalence of IT communications, vertical and

horizontal, often informal

a8 Greater competency-based autonomy

Assessment / P b1 Novice

b2 Specialist

b3 Experienced specialist

b4 Expert

b5 Master

Learning activities / <C, A, F> c1 Classroom training

c2 E-Learning

c3 Blended learning

c4 Social learning

c5 Modern workplace learning

A separate problem is matching of the rest parameters, namely "Learning activities" according to (1) and <educational content, learning tools, form of training> according to (2), with workplace learning realities in HSOC. As noted above and as confirmed by the analysis of literary sources [4, 21] and the practice of HSOC, there is a lot of forms of workplace learning applicable to HSOC, which can be classified for various reasons. In our work, we use the classification proposed in [4], where the variants for organizing and implementing the educational process in workplace learning are considered as historical stages of its development. This approach has several advantages that are

confirmed in the literature. First, the development of workplace learning went along the path of strengthening the informal component, i.e. the classification used simultaneously describes the level of formalization of a specific approach to overhead lines. Secondly, in modern practice, all these approaches exist and are used in parallel.

Thus, we consider the three parameters <C, A, F> from model (2) as a strongly connected complex correlated with learning activities from model (1) and interpret it as stages of workplace learning according to [4]. These are presented in Table 1 (low row).

In order to identify workplace learning ways that best provide the needed competencies of HSOC personnel, we base on the rough set theory [25], which allows you to select from a common set of objects (defined by a set of attributes) a subset of objects similar in a certain sense. We use a modification of the theory of rough sets proposed in [26], according to which attributes are given a meaningful sense of the degree of equivalence relations between objects, which can be ranked on any scale.

Belonging to the equivalence class for relations between each pair of objects was determined by experts on a 4-point scale of the level of connectivity (compliance): 1 - high; 2 - medium, 3 - low, 4 -weak / absent. The expert group included heads of IT departments of the HSOC, HR HSOC specialists and specialists in additional professional training, for a total of 8 people. The results of expert assessments of the relationships between the parameters of the constructed model (Table 1) are presented in Table 2 (reflecting the relationship between Learning outcomes and Learning activities) and Table 3 (reflecting the relationship between Learning outcomes and Assessment).

Table 2. Expert assessment of equivalence relationship between Learning outcomes and Learning

activities.

a1 a2 a3 a4 a5 a6 a7 a8

c1 2 4 3 3 1 3 4 4

c2 3 4 2 2 1 3 2 4

c3 2 3 1 3 3 3 2 3

c4 1 2 2 2 3 2 1 2

c5 2 2 3 1 1 1 1 2

Table 3. Expert assessment of equivalence relationship between Learning outcomes and Assessment.

a1 a2 a3 a4 a5 a6 a7 a8

b1 4 4 4 4 4 3 4 4

b2 3 4 3 3 3 3 3 4

b3 2 4 3 3 2 2 3 4

b4 1 2 2 1 1 2 2 1

b5 1 2 1 1 1 1 2 1

Based on the data in table 2, we calculated the average value of the level of equivalence for all possible groups of parameters a1-a8. To solve the computational subtasks associated with the application of the theory of rough sets, a program was written in Python. The program implements the functions pos, IND, gamma (importance), as well as parsing the data of tables 2 and 3, which are the input data for the calculation. The program enumerates all combinations of attributes a1-a8 to divide the universe of table 2 into subgroups of closely related attributes. To select significant subgroups, the resulting subgroups are sorted by the length of the attribute vector (4-3-2-1) and the arithmetic average of equivalence level for each subgroup (avg) is calculated. The most significant are considered the subgroups of the greatest length with the smallest value of the average level of equivalence level. The program code for the calculation is available at the link1.

The calculation results are presented in Figure 1. The figure outlines those groups of parameters for which the average value of the degree of equivalence is minimal, i.e. they have the closest relationship. Of most interest, obviously, is the group with avg = 1.67 as the largest. In accordance with table 2, it includes three values of the parameter "Learning outcomes", namely a2, a7, a8, and they correspond to the values c4 and c5 of the "Learning activities" parameter. From table 3 we see a similar correspondence between the parameters a2, a7, a8 and b4, b5 reflecting the "Assesment" parameter.

Combining the obtained results, we can state that the competencies a2, a7, a8 are most important for employees at the Professional (b4) and Expert (b5) levels, and for the development of these

1 https://github.com/boyvita/rough sets

competencies in them, it is advisable to apply the approaches Social learning (c4) and Modern workplace learning (c5). At the same time, for employees of Specialist, Experienced specialist and Novice levels, this group of competencies is not representative, and for them applying the above training methods is inappropriate.

[2, 5, 6, 8]: [[1, 4]j [2], [3], [5]] groups 4]:4234 avg=3.25;

[1, 4j 6]: [[1, 3], [2]j [4], [5]] group[l, 3] 233 avg=2 67;

[2j 5, 6]: [[1, 4], [2]j [3], [5]] group[l, 4] 423 avg=3 00;

[2, 5, 8]: [[1, 4], [2], [3], [5]] group[l, 4] 424 avg=3 33;

[2, 6, 8]: [[1, 4], [2]j [3], [5]] group[l, 4] 434 avg=3 67;

[5, 6, 8]: [[1, 4], [2]j [3], [5]] group[l, 4] 234 avg=3 00;

[2j 1, 8]: [[1], [2j 5], [3], [4]] group[2j 5] 212 avg=l 67;

[1, 3] [[1, 5], |[2L [3] , [4]] group[l, 5] 23 avg= =2 50

[1, 4] [[1, 3], |[2L [4] , [5]] group[l, 3] 23 avg= =2 50

[3, 4] [[1], [2j 4]j [3] , [5]] group[2, 4] 22 avg= =2 00

[2, 5] [[1, 4L [2], [3] , [5]] group[l, 4] 42 avg= =3 00

[1, 6] [[1, 3], |[2L [4] , [5]] group[l, 3] 23 avg= =2 50

[2 j 6] [[1, 4], |[2L [3] , [5]] group[l, 4] 43 avg= =3 50

[4j S] [[1, 3], [2]j [4] , [5]] group[lj 3] 33 avg= =3 00

[5, 6] [[1, 4], |[2L [3] , [5]] group[l, 4] 23 avg= =2 50

[2 j 7] [[1], [2j 5], [3] , [4]] group[2, 5] 21 avg: =1 50

[6, 7] [[1], [2]j [3, 4] , [5]] group[3, 4] 32 avg= =2 50

[2, 8]: [[1, 4]j [2, 5]j [3]] groupCl, 4]:44 avg=4.00; group[25]:22 avg=2.00;

[5, 8]: [[1, 4], [2] j [3], [5]] group[l, 4]:: 24 avg=3.00;

[6, 8]: [[1, 4], [2] j [3], [5]] group[l, 4]:: 34 avg=3.50;

U, B]: [[1], [2, 5]j [3], [4]] group[2, 5]:12 avg=1.50;

Figure 1. Calculation results. *Output format: [attribute group for partition]: [departed object groups] *followed by max-sized groups * [group[result group]]: *attributes values within the group*

avg = average values for a group]*

A substantial analysis of the parameters c4 (Social learning) and c5 (Modern workplace learning) introduced in [4Hart] shows that they most closely correspond to the triple ""On the job / structured / passive" from model (3), that is, learning process should occur at the actual work setting as a result of using a systems approach, and with limited involvement of a trainer/facilitator [21]. Comparing our findings with the results of the 2019 survey [4] mentioned in the Introduction, we can argue that the ways of workplace learning preferred by top-level specialists, namely (1) daily work experience, (2) knowledge sharing within the work team, (3) personal and independent search on the Internet, (4) feedback from the leader / mentor, are fully consistent with this triple of parameters.

Thus, in the section, a technique for identifying ways of workplace learning that best provide the needed competencies of HSOC personnel is constructed, and its effectiveness is confirmed by comparison with independent survey results.

3 DEVELOPMENT OF MEANS OF SUPPORTING NON-FORMAL WORKPLACE LEARNING FOR HSOC PERSONNEL

In the previous section, we identified the most effective ways for workplace learning of top-level HSOC employees, which reduce the need for personal contacts of the employee with the facilitator and allow indirect control of the process of non-formal learning. However, an analysis of the literature has shown, methods and software tools for supporting this kind of workplace learning are very weakly presented in the literature and in real practice. The set of such means that we developed to support workplace learning for top-level HSOC employees that meets the above requirements is presented below.

3.1 Means of teaching personnel in a real-work environment

As a means of supporting non-formal workplace learning of personnel, a method has been developed for teaching how to configure network infrastructure, which provides a more complete immersion of users in a real-work environment [27]. The architecture of the system implementing the method is depicted in Figure 2. Unlike traditional packages for networking configuration (for example, Cisco

Packet Tracer2), the developed system works directly in a real environment where the employee carries out the development, modeling, debugging and verification of the network configuration as his production task. This avoids errors when manually migrating configurations. An additional advantage of the developed system as a means of training is that here the employee is faced with various artifacts that may occur in a real work environment, i.e. he is forced to process poorly defined information and thereby gain valuable experience.

We fulfilled an experimental test of the effectiveness of the developed system for user groups with different levels of training: (A) students of technical specialty before studying a course "Administrating of information networks", (B) students of technical specialty after studying a course "Administrating of information networks", (C) teachers of IT disciplines, (D) system engineers.

During experiments we compared the runtime of the tasks and the type of the mistakes made when working in the proposed system and in the Cisco Packet Tracer. In all groups, the runtime in the developed system was shorter than in Cisco, and this effect was more pronounced in weaker groups (A and B). A similar relationship was identified with respect to the total number of errors made. At the same time, weak groups were dominated by errors related to the level of formal knowledge (data entry errors, errors of a choice of interfaces, physical lines, etc.). Such errors in strong groups almost disappeared, and errors of interpretation of environmental behavior (security policy, time scale, need to reset of elements of the model, etc.) prevailed here.

Thus, the conducted experiments have confirmed that the developed system provides an adaptive and intelligent environment for workplace learning of system engineers which are an important component of the HSOC staff.

HOST N

/ Linux Network Namespace #1 \ Services for client #1

Figure 2. Architecture of the system for teaching how to configure network infrastructure

3.2 Means of answering employees' professional questions

For answering employees' professional questions the dialog system [28] aiming to support users developing the embedded applications based on the OS QNX Neutrino RTOS v. 6.53 was built. The system contains taxonomic information from technical manual4 provided by the company, as well as

2 https://www.packettracernetwork.com/

3 http://blackberry.qnx.com/en/products/neutrino-rtos/neutrino-rtos

4http://www.qnx.com/developers/docs/6.5.0/index.jsp?topic=%2Fcom.qnx.doc.neutrino%2Fbookset.html

non-taxonomic expert information obtained from the most competent technical specialists. To aggregate such heterogeneous information, the methods of formal concept analysis [29] and of relational concept analysis were used. A fragment of the corresponding cross-table is presented in Table 4, where concepts gi correspond to information extracted from the official manual, and concepts mj reflect expert information. Thus the a concept lattice for the entire domain was built which additionally took into account the weights of the relations between the concepts. Based on this lattice the service desk dialog system is managed, which took on the role of level 0 in a three-tier support service built in accordance with the principles of ITIL5. The architecture of the Level 0 subsystem is shown in Figure 3.

Table 4. Fragment of cross-table

R m1 m2 m3 m4 m5 m12 m14 m15 m16 m17

g1 *

g2 * *

g4 * *

g8 * *

g9 * * * * * * * *

g12 * *

g14 * * * *

Figure 3. Architecture of the Level 0 subsystem

Experimental estimates showed that the system allows in automatic mode, i.e. without contacting the facilitator, process 52% of incoming questions, and for questions of the HELP category that are most important for workplace learning in HSOC, this value increases to 68%.

3.3 Means of estimating employee web-behavior

Searching the Internet is an integral part of the activities of the majority of HSOC employees. The range of topics that an effective employee chooses to support his workplace learning can be considered a reference set, and vice versa. To monitor the effectiveness of the employee's learning process based on their current activity on the Internet, we have developed a method for estimating employee behavior deviations from effective Internet search scenarios [30]. In order to do this, we form "behavioural portraits" of effective ("ideal") and ineffective ("real") employees via topic modelling. We used machine learning methods, namely latent Dirichlet allocation (LDA), enhanced by additive regularization of thematic models (ARTM) [31].

In order to model the web-browsing of an effective ("ideal") and an ineffective ("real") employee, we built datasets based on the Kos articles collection6 (see [30] for a detailed description of the procedure). For topic modeling, we have chosen the BigARTM library7.

We experimentally investigated the efficiency of comparing the portraits of the ideal and real employee using ARTM or LDA models. The results are presented in Figure 4, where It indicates belonging of the article chosen to the topic t. From the graph it can be seen that both models are acceptable for monitoring the effectiveness of an employee's search strategy during workplace learning. The LDA model better identifies the strong deviations of the employee from the basic search topic, and the ARTM model works better within the topics closer to the basic one.

5 ITIL v3 2011 (IT Infrastructure Library v3 2011 Edition

6 dailykos.com

7 http://bigartm.org/

Figure 4. Dependence of It on topic probability and its variance for LDA (black lines)

and ARTM (gray lines) models

The method proposed can be used by employees in order to build their own effective search strategy within non-formal workplace learning process, as well as for create a corpus of information and training materials for training new employees or for selecting content on a desired topic.

4 RESULTS AND DISCUSSION

The developed methods and systems were used in workplace learning of key technical staff in Rotek LLC (the IT-company providing outsourcing services of printing, office equipment and communication equipment) for six months. The set of key performance indicators (KPI) evaluated during this period and the change in KPI compared to the previous six month period is shown in table 5.

Table 5. The changes of KPIs

No. Title Change

K1 Average customer rating of service +37%

K2 Average length of service interruptions -21%

K3 Number of service interruptions -19%

K4 Average length of new services deployments -34%

K5 Number of recurring incidents for which a solution method already exists -47%

K6 Number of Incidents resolved remotely +49

K7 Number of incidents resolved during the first contact or resolved by the automated dialog system +51

K8 Rate of incidents resolved during solution times agreed in SLA +54

The obtained improvement in key performance indicators was from 19 to 54%. The results obtained allow us to confirm that:

- the efficiency of customer support has increased,

- the time to provide new services has been reduced,

- the reliability of services has increased,

- the overall customer satisfaction increased.

Significant improvements in all KPIs suggest that the developed complex of methods and software systems implementing them can be applied directly at employees' workplaces as part of the implementation of real production tasks, which creates effective conditions for managing the training of HSOC personnel based on the results of their current professional activity.

5 CONCLUSIONS

All the objectives of the research were achieved.

- Based on the analysis of the required competencies of HSOC stuff suitable ways of workplace learning that best provide the necessary competencies of HSOC stuff were identified. In particular, important competencies for Professional and Expert level stuff and the suitable approaches to the organization the workplace learning were identified.

- A set of methods and software tools for support workplace learning has been developed. These methods and tools allow to reduce the need of permanent personal contacts between the trainee and the facilitator and allow indirect control of the informal learning process.

The experiment showed that the use of research results positively affects to the results of workplace learning of HSOC's stuff.

ACKNOWLEDGEMENTS

This work was financially supported by Russian Federation, Grant 08-08.

REFERENCES [ARIAL, 12-POINT, BOLD, LEFT ALIGNMENT]

[1] A. Zakrzewska-Bielawska "High technology company - concept, nature, characteristics".

Proceedings of the 8th WSEAS Int. Conf. on Management, Marketing and Finance, WSEAS Press, Penang, Malaysia, March 23-23 2010, pp. 93-98

[2] M. Wolf and D.Terrell, "The high-tech industry, what is it and why it matters to our economic future," Beyond the Numbers: Employment and Unemployment, vol. 5, no. 8 (U.S. Bureau of Labor Statistics, May 2016)

[3] K. Cacciattolo. "Defining workplace learning". European Scientific Journal May 2015 /SPECIAL edition. Vol.1. Pp. 243- 250

[4] J. Hart. "Modern Workplace Learning 2019: A Framework for Continuous Improvement, Learning and Development". [Online]. Retrieved from https://www.modernworkplacelearning.com/cild/

[5] F. Sweeney. "Workplace learning: The evolution of IT learning". March 2016. [Online]. Retrieved from https://www.theceomagazine.com/business/management-leadership/workplace-learning-the-evolution-of-it-learning/

[6] J. Dieffenbach, C. Diemand-Yauman." Workplace Learning: What's It Worth?" April 2019. [Online].

Retrieved from https://www.forbes.com/sites/civicnation/2019/04/23/workplace-learning-whats-it-worth/#a3d83cf25c34

[7] S. Ghosh. "The Future of Workplace Learning: Top Trends and Predictions for 2019-2020". Febr. 2019. [Online]. Retrieved from https://indecommdigital.com/insight/the-future-of-workplace-learning-top-trends-and-predictions-for-2019-2020/

[8] Workplace Learning Report 2019 - LinkedIn Learning. [Online]. Retrieved from

https://learning.linkedin.com > amp > images > pdf

[9] N. Galanis, E. Mayol, M. Alier, F.J. Garcia-Penalvo. "Designing an Informal Learning Support Framework. 2015". [Online]. Retrieved from https://core.ac.uk/download/pdf/41824932.pdf

[10] C. Toure, C.Michel, J.-C. Marty. 2017. "How to Promote Informal Learning in the Workplace? The Need for Incremental Design Methods". [Online]. Retrieved from https://arxiv.org/abs/1709.09945

[11] F. J. Garcia-Penalvo, M. A. Conde (2014). "Using informal learning for business decision making and knowledge management". Journal of Business Research, 67(5), 686-691.

[12] C. Lellis. "10 Ways Technology Can Power Workplace Learning". March 2018. [Online]. Retrieved from http://www.perillon.com/blog/10-ways-technology-can-power-workplace-learning

[13] G. Conole. "Review of pedagogical models and their use in e-learning". 2010. Retrieved from https://tecfa.unige.ch/tecfa/teaching/formcont/certificatElearning/Mod2cours2/pedmodelsanduseE l.pdf

[14] V.J. Marsick, K.E. Watkins. "Informal and Incidental Learning". In: New directions for adult and continuing education, no. 89, Spring 2001. Jossey-Bass, A Publishing Unit of John Wiley & Sons.

[15] Peng Lu et al. The Research on Informal Learning Model of College Students Based on SNS and Case Study (2017). J. Phys.: Conf. Ser. 820 012025

[16] M.C. Pettenati, M. Ranieri. Informal learning theories and tools to support knowledge management in distributed CoPs. In: E. Tomadaki and P. Scott (Eds.). Innovative Approaches for Learning and Knowledge Sharing, EC-TEL 2006 Workshops Proceedings, ISSN 1613-0073, p. 345-355, 2006.

[17] Biggs, J. and Tang, C. (2011): Teaching for Quality Learning at University, (McGraw-Hill and Open University Press, Maidenhead)

[18] Hodkinson, P., & Hodkinson, H. (2004). "The complexities of workplace learning: Problems and dangers in trying to measure attainment". In H. Rainbird, A. Fuller, & A. Munro (Eds.), Workplace learning in context (pp. 259-275). London: Routledge.

[19] Sambrook, S. (2005). "Factors influencing the context and process of work-related learning: Synthesizing findings from two research projects". Human Resource Development International^, 101-119.

[20] Simonov V.P. Educational Management: Know How in Education. Moscow, Vysshee Obrazovanie Publ., 2006, 368 p. (In Russian)

[21] R.L. Jacobs. A proposed conceptual framework of workplace learning. Human Resource Development Review. Vol. 8, No. 2 June 2009. Pp. 133-150

[22] L. Prifti, M. Knigge, H. Kienegger, H. Krcmar. "A Competency Model for "Industrie 4.0" Employees". 2017. Retrieved from https://www.wi2017.ch/images/wi2017-0262.pdf

[23] D. Russo. "Competency Measurement Model". European Conference on Quality in Official Statistics (Q2016) Madrid, 31 May-3June 2016

[24] The Competency Framework. A guide for IAEA managers and staff. Retrieved from https://www.iaea.org/sites/default/files/18/03/competency-framework.pdf

[25] Zhang Q., Xie Q., Wang G. "A survey on rough set theory and its applications". CAAI Transactions on Intelligence Technology. Volume 1, Issue 4, October 2016, Pp. 323-333

[26] O. Uzhga-Rebrov. "Knowledge representing features in rough sets theory". Environment. Technology. Resources Proceedings of the 7th International Scientific and Practical Conference. Volume II, Rezeknes Augstskola, Rezekne, RA IzdevniecTba, 2009.

[27] Artemova G.O., Beresnev A.D., Buraia K.I., Gusarova N.F. "System of Network Infrastructure Configuring for Project Learning". Journal of Engineering and Applied Sciences - 2017, Vol. 12, No. 9 SI, pp. 8607-8613

[28] Beresnev A., Zhdankin A., Lobantsev A., Vasiliev A., Vedernikov N., Gusarova N. "Dialogue System for Service Desk of Complex Software Systems Based on Relational Concept Analysis". ACM International Conference Proceeding Series - 2019, Vol. F148261, pp. 31-36

[29] J. Poelmans, D. I. Ignatov, S. Kuznetsov, G. Dedene "Formal concept analysis in knowledge processing: A survey on applications". P. 2. Expert Systems with Applications. 2013. Vol. 40. No. 16. P.6538-6560.

[30] Vatian A., Dudorov S., Beresnev A., Vasilev A., Nigmatullin N., Vedernikov N., Stankevich A., Gusarova N., Shalyto A. "Topic modeling of text content for monitoring the employee's efficiency via his internet activity". Multi Conference on Computer Science and Information Systems; Proceedings of the International Conferences on Big Data Analytics, Data Mining and Computational Intelligence 2018, Theory and Practice in Modern Computing 2018 and Connected Smart Cities 2018, MCCSIS 2018 - 2018, pp. 43-50

[31] Vorontsov K.V., Potapenko A.A., 2015." Additive regularization of topic models". Mach.Learn. Special Issue on Data Analysis and Intelligent Optimization with Applications Vol. 101(1), pp. 303-323

Comparison of Intelligent Classification Algorithms for Workplace Learning System in High-Tech Service-Oriented Companies

Artem Beresnev1 and Natalia Gusarova1,

1 ITMO University, 49 Kronverksky Pr, St. Petersburg, Russia [e-mail]

Abstract. We investigate the characteristic of several intelligent algorithms for the program dialogue module of the support system of development personnel of high-tech service-oriented companies. Briefly describes the parametric model of workplace learning as base for personnel development and the most appropriate approaches to the formation of specific staff competencies. One of the elements of the proposed system is the means of answering personnel professional questions. In such applications, an important role is played by means of preliminary classification of queries that allow to narrow the search domain and increase the relevance. Three approaches to classifying of questions were compared: The Naive Bayes classifier, Random Forest Classifier and neural network. A comparative assessment of such approaches was carried out.

Keywords: dialogue system, text classification, naive Bayes, random forest, neural network, small dataset.

1 Introduction

Modern high-tech companies build their business around providing services to the customer, rather than selling the product to the customer. This property gives them significant advantages in the market [1, 2]. We will call such companies high - tech service-oriented companies (HSOC). The market for high-tech services in most niches is characterized by high competition, and significant dynamics of technology change. Therefore, both the composition of the service and the quality of its providing are very important. Therefore, HSOC must pay considerable attention to the development of the staffs technological competence in the latest industry solutions, innovation activity and communication skills. This is especially relevant for small and medium-sized enterprises and startups, for which the contribution of an individual employee to the final success is significant.

Different approaches are used - professional courses, conferences, e-learning, and workplace learning. The latter is obviously useful if the service of a company has its specifics or a high rate of change.

In previous work [3], the question of the structure, specifics, and composition of the workplace learning was discussed in detail. Briefly review the basic concepts, provisions, and conclusions, which are necessary for this paper.

In the research [4], workplace learning is divided into formal and informal. Formal methods include conferences, refresher courses, using in-house tutorials, and so on. Informal workpalce learning is based on the employee's personal initiative and can take the form of communication with colleagues or a team leader, Web-searfing and experiment in a model or real environment. These acts arise in connection with current work tasks. Therefore, informal workplace learning is often sporadic and, unlike formal workplace learning, does not have a pre-defined form and structure.

Surveys show that HSOC's staff prefer an informal workplace learning with such approaches as daily work experience, knowledge sharing within the work team or the leader, and personal Web-searching [4]. At the same time, researchers note the existence of pedagogical, qualification, and resource problems for informal workplace learning [4-7]. In our work [3], we have identified ways of informal workplace learning that provide the necessary competencies for HSOC staff and developed methods and software tools to support such ways of workplace learning, which remove some recourses restrictions and allow provide indirect control of the informal learning process.

To identify the mentioned methods and software support tools, we used the analysis of a combined parametric model based on several models of pedagogical processes:

— parameterizing model of the pedagogical process [8]:

Learning process =<Learning activities, Learning outcomes, Assessments, (1)

— model for pedagogical system [9]:

PS = <T, P, C, A, F>, (2)

— and workplace learning model proposed in [10]:

CF = < Location of the learning (Off the job / On the job); Degree ofplanning (Unstructured / Structured); Role of trainer/facilitator (Passive/Active) >. (3)

Note that model (3) is a conceptual framework of workplace learning.

In [1], we correlated these models and, using rough sets methods and content analysis, identified means to support workplace learning for HSOC. These include means of teaching personnel in a real-work environment, means of estimating employee web-behavior, means of answering employees' professional questions.

In last means the methods of formal concept analysis and of relational concept analysis were used and good results were achieved [11]. At the same time, the approach with automatic classification of requests, that is common for knowledge base and Help Desk systems was not used. Such classification involves separating requests by a predefined set of classes <K>. Each element Ki is an entity that corresponds to the subject area. These can be supported services, service components in any superposition (subsystems, programming languages, frameworks, role's administrative authority, stage of business process etc.). In the task statement [11], Ki is an additional attribute for

searching for a facilitator for consulting or expert to actualize an ontological model. However, in the developed system, this approach was not used in its pure form, and there are reasons to believe that its implementation may be appropriate. Thus, the task of this paper includes:

— identifying the specific of workplace learning processes for HSOC, related to initial requests processing in the dialog module,

— identification the specific of the existing datasets,

— identifying the main suitable approaches and algorithms for classification,

— identification of the most suitable algorithms for specific conditions.

Applying Classification Approaches for Means of Answering Employees' Professional Questions

2.1 General Information and Main Problems

We are going to support an employee in a typical situation processing with customer requests. These requests are given in the form of short text messages. Another case described in detail in [11] is the process of answering professional questions from employees in the internal dialog system. Both approaches can be organized as a three-level support service built in accordance with ITIL1 principles. In [11], the automatic system was introduced as Level 0, which analyzed incoming requests and found the most appropriate sections of the knowledge base and reference system. Now we propose to modify this module to transmit to the Level 1 both recommendation and an automatically defined class Ki. This information extends the ability to support the work of the employees and serves as implicit workspace learning. The proposed change to the means of answering employees' professional questions architecture is shown in Fig. 1.

To Ley/el 1

To Level 1

Lattice Module

Language Parser Ç—^

Knowledge Base

LEVELO

Knowledge Base

Decision Module

To Consulting System

Lattice Module )

Language Parser Í )

Decision Module

To Consu Itlng System

Classifier subsystem

Fig. 1. Proposed modification of the Level 0 subsystem architecture.

There are many papers [12-15] devoted to the comparison of intelligent algorithms for text classification. However, research have been realized for datasets with certain characteristics. Such datasets are characterized by both a large volume (more than 100,000 text units) and a certain context (reviews of products or movies, requests for rescue,

1 ITIL v3 2011 (IT Infrastructure Library v3 2011 Edition

2

etc.). The context of each message is precisely defined by external attributes (product or movie id, geotags, timestamps, hashtags). Another important feature of these research is the use of open data with free access. The results of these research can be used only in a limited way for the purpose of this paper, just in the case of large and superlarge HSOC. Such organizations already have long-accumulated data sets in Help Desk and Service Desk systems, where text units often are already reliably and correctly attributed during common work processing.

At the same time for small and medium HSOC typical:

— need to work with small datasets (2-3 thousand records),

— short length of each individual unit in the dataset (about fifty words),

— inability to use external specialized datasets due to legal requirements or commercial secrecy considerations,

— significant impact of erroneous object markup due to the high relative significance of an individual object in whole dataset,

— strong influence of implicit factors. For example, when different text units containing similar or identical sets of keywords are assigned to different classes Ki by the expert, based on the expert's knowledge, that is, based on data structures that are not present in the system at all,

— presence of "garbage" keywords in text units, for example, when the author of a text message makes suggestions on a topic.

All this makes important to compare the main intelligent text classification algorithms in a case of small and medium HSOC. In addition, there are language dependency, contextual dependency, and social dependency, but these issues are beyond the scope of this paper.

2.2 The Specificity of The Dataset

We used an open dataset2. This dataset is a set of messages to first line technical support. Each case is represented as a text block. The dataset contains 3000 records. Each entry is marked with one of five categories - Ki. As shown in Table 1, the data set is evenly distributed across categories.

Table 1. The number of text blocks per categories

Categories Number of text blocks

Application 600

Database 600

Network 600

Security 600

User Maintenance 600

2 https://github.com/karolzak/support-tickets-classification#22-dataset

In total

3000

Text blocks have a length of 2 (!) up to 927 words, including prepositions, articles, and so on. The distribution of the number of text blocks by length is shown in Fig. 2.

400 500 500 700 Le rigth of th e te xt bl ock

Fig. 2. Distribution of text blocks by length.

It is obvious that the main part of messages are text blocks about 25 words long. Moreover, even very short messages (2 - 5 words long) have significant content and cannot be discarded (for example: "hello please fill date", "forgot cable return", "access sa"). At the same time, different text blocks containing similar keywords have different categories. For example: "Hello, please provide a valid license as soon as possible appears my expired thank you engineer" as a Database, "Hello dear requires a Visual Studio license the ability to provide considers engineer mob" as an Application.

The analysis is complicated, because the text blocks contain the author's name, words in Spanish, and a description of several tasks in one block.

The analysis shows that the investigated dataset is suitable for the characteristics given in 2.1 and can be used to compare the main intelligent text classification algorithms in a case of small and medium HSOC.

3 Method and Material

3.1 Experiment Methodology

Three intelligent text classification algorithms were selected for comparison: Naive Bayes classifier, Random Forest Classifier and Neural Network. These algorithms are independent of natural language. This is important because the situation of working in

a heterogeneous language environment is typical for HSOC (for example, to provide services in foreign countries or localize software). A special program in Python was developed to compare the algorithms. The experiment was performed in the following steps:

- two parts were selected from the original dataset - one for training algorithms, and the second for testing. The division is carried out in a ratio of 1 to 9 for each of the predefined Ki classes. Thus, the training sample was only 300 messages.

- It was performed traditional text preprocessing. Then all text blocks were represented as vectors of numbers.

- All three algorithms were trained. The program code implements the train_model() function that passes data to the desired algorithm.

- All algorithms were checked on the test part of the dataset. For each case, the accuracy of determining the message class Ki was calculated. Comparison of accuracy values is used to compare the efficiency of algorithms in specified conditions.

3.2 Data Preprocessing

Text preprocessing contained converting text to lowercase, removing prepositions etc. Significant entities and abbreviations were preserved during the cleaning. The shortest messages (up to 5 words) were checked manually, and obviously incorrect ones were deleted.

It was necessary to represent text blocks as vectors of numbers that reflect the importance of each word. To calculate the weights of words we used a combined measure of the importance of Tf-Idf (term frequency and inverse document frequency).

tf - idf(t, d, D) = tf(t, d) x idf(t, D) (4),

t№d)=ik (5) idf(t- = <6>

Where nt is the number of occurrences of the word t in document d from the collection of documents D. In our case, D is the set of all text blocks.

3.3 Naive Bayes Classifier

In this paper, we used the implementation of the Naive Bayesian Classifier-naivebayes from the sklearn library. The Naive Bayesian Classifier is a simple probabilistic classifier based on the Bayes theorem [16]. It is important that properties are considered independent. The Ki class of the text block was determined by the maximum probability value (7). Where xi is the property vector.

K = argmaxi=l k(p(Ki) p(xj %)) (7)

3.4 Random Forest Classifier

Random Forest Classifier is an algorithm that uses an ensemble of decision trees. Classification is carried out by voting. Each tree assigns an object to one of the classes, and the class that the largest number of trees voted for wins [17]. To implement the classifier, we use the sklearn library again.

3.5 Neural Network

Neural networks are widely used for solving classification problems [18]. We used Feed-forward Neural Network because implementations of more complex structures such Recurrent, Convolutive or Long short-term memory networks are experimental and complicated for quick realization [12, 18]. The network architecture is shown in

Fig. 3.

dense_ 1 _input: Input Layer input: (None, 2069)

output: (None, 2069)

dense_J: Dense input: (None, 2069)

output: (None, 500)

1 r

dense_2: Dense input: (None, 500)

output: (None, 1)

Fig. 3. The neural network architecture.

The resource and time constraints on retraining that are typical for small HSOC made us choose such a simple architecture. Due to these limitations, we used the activation function ReLU (rectified linear unit) since it is devoid of resource-intensive operations and provides fast learning. For the same reasons, the early stop method was chosen to determine the number of learning epochs. Network performance comparisons were made every 5 epochs. As a result, it was found that after 100 epochs, the network performance drops and it goes into a state of retraining.

4 Results

Under the described conditions, all three algorithms were trained, and accuracy was calculated on the test part of the datasets. Accuracy is the proportion of messages by which the classifier made the right decision. It is important to note that accuracy is not often used to evaluate a Naive Bayes classifier, but in our dataset, the number of messages of each class is the same, so accuracy is applicable in this case. The results of the

accuracy calculation for each of the three text classification algorithms are shown in Table 2. They are ordered according to the decreasing result.

Table 2. Result of comparison.

Algorithm Accuracy

Naive Bayes classifier 0,748

Random Forest Classifier 0,722

Neural Network 0,694

As one can see, nearly accurate values were obtained for the compared algorithms, bit Naive Bayes showed slightly better results. In addition, Naive Bayes is characterized by simplicity in comparison with other classification algorithms, which allows Naive Bayes to better adapt to working with small, low-quality datasets. Indirectly, the reliability of the results is confirmed by the fact that similar accuracy values were obtained by other researchers who worked with small datasets [19].

5 Conclusions

All the objectives of the research were achieved.

The specifics of workplace learning processes for small HSOC associated with the initial processing of requests in the dialog module were identified. This specificity lies in the fact that for small HSOC, the use of the dialog system [3] corresponds to the content of the activities and preferences of HSOC employees, at the same time, the capabilities of the dialog module can be expanded by additional classification of message texts, for example, to determine the topic or find the most suitable facilitator.

Features of typical datasets for small HSOC were identified. It was shown that data sets with which small and medium-sized HSOCs start working are characterized by a small volume (2-3 thousand records) with a small length of each the records themselves, a strong influence of erroneous unit markup due to the high specific significance of a single unit, a strong influence of implicit factors, and the presence of garbage keywords in units. In addition, it is difficult or impossible to access ready-made data sets of other owners.

The following set of suitable approaches and algorithms for text classification was selected: Naive Bayes classifier, Random Forest Classifier, and Neural Network classification. The selection criteria were independence from the natural language, undemanding access to resources, quick start, and widespread support for methods in common software libraries.

Among these classification algorithms, slightly better results are obtained for Naive Bayes and it provides higher reliability in low quality datasets conditions.

As a further work, we plan to realize the results clarification with other similar datasets. The influence of the dataset specificity on the results obtained is obvious. Therefore, additional experiments will allow us to make more general conclusions.

Acknowledgements

This work was supported by Russian Science Foundation, Grant #19-19-00696.

References

1. Zakrzewska - Bielawska, A.: High technology company - concept, nature, characteristics. In: N. Mastorakis, V. Mladenov, A. Zaharim, C. Aida Bulucea (eds.). RECENT ADVANCES IN MANAGEMENT, MARKETING, FINANCES 2010, pp. 93-98. WSEAS Press, Penang, Malaysia (2010).

2. Wolf, M., Terrell, D.: The high-tech industry, what is it and why it matters to our economic future. Beyond the Numbers: Employment and Unemployment, vol. 5, no. 8. Bureau of Labor Statistics. Washington, DC (2016).

3. Beresnev, A.D., Boytsov, V.V., Dobrenko, D.A., Egorov, N.V., Gusarova, N.F. Workplace Learning For Personnel Of High-Tech Service-Oriented Companies. In: 12th annual International Conference of Education, Research and Innovation, pp. 9693-9703. IATED, Seville, Spain. (2019).

4. Hart, J.: Modern Workplace Learning 2019: A Framework for Continuous Improvement, Learning and Development, https://www.modernworkplacelearning.com/cild/, last accessed 2020/02/15.

5. Sweeney, F.: Workplace learning: The evolution of IT learning. https://www.theceomaga-zine.com/business/management-leadership/workplace-learning-the-evolution-of-it-learn-ing/, last accessed 2020/02/18.

6. Dieffenbach, J., Diemand-Yauman, C.: Workplace Learning: What's It Worth? https://www.forbes.com/sites/civicnation/2019/04/23/workplace-learning-whats-it-worth/#a3d83cf25c34, last accessed 2020/03/01.

7. Ghosh, S.: The Future of Workplace Learning: Top Trends and Predictions for 2019-2020, https://indecommdigital.com/insight/the-future-of-workplace-learning-top-trends-and-pre-dictions-for-2019-2020/, last accessed 2020/03/01.

8. Biggs, J., Tang, C.: Teaching for Quality Learning at University. McGraw-Hill and Open University Press, Maidenhead (2011).

9. Simonov, V.P.: Educational Management: Know How in Education. Vysshee Obrazovanie Publ., Moscow (2006).

10. Jacobs, R.L.: A proposed conceptual framework of workplace learning. Human Resource Development Review 2(8), 133-150 (2009).

11. Beresnev, A., Zhdankin, A., Lobantsev, A., Vasiliev, A., Vedernikov, N., Gusarova, N. Dialogue System for Service Desk of Complex Software Systems Based on Relational Concept Analysis. In: Proceedings of the 2019 2nd International Conference on Geoinformatics and Data Analysis 2019, vol. F148261, pp. 31-36. Prague (2019).

12. Romanov, A., Kozlova, E., Lomotin, K.: Application of NLP Algorithms: Automatic Text Classifier Tool. In: Alexandrov, D.A., Boukhanovsky, A.V., Chugunov, A.V., Kabanov, Y., Koltsova, O. (eds.) DIGITAL TRANSFORMATION AND GLOBAL SOCIETY, THIRD

INTERNATIONAL CONFERENCE 2018, Part II, pp. 310-323. St. Petersburg, Russia (2018).

13. Alharbi, A.N., Alnnamlah, H., Liyakathunisa: Classification of Customer Tweets Using Big Data Analytics. In: Alenezi M., Qureshi B. (eds) 5TH INTERNATIONAL SYMPOSIUM ON DATA MINING APPLICATIONS. ADVANCES IN INTELLIGENT SYSTEMS AND COMPUTING, vol 753, pp. 169-180 Springer, Cham (2018).

14. Hong, Y., Sinnott, R.O.: A Social Media Platform for Infectious Disease Analytics. In: Gervasi O. et al. (eds) Computational Science and Its Applications - ICCSA 2018. ICCSA 2018. Lecture Notes in Computer Science, vol 10960, pp. 526-540. Springer, Cham (2018).

15. Agogo, D., Hess, T.J.: Scale Development Using Twitter Data: Applying Contemporary Natural Language Processing Methods in IS Research. In: Deokar, A., Gupta, A., Iyer, L., Jones, M. (eds) ANALYTICS AND DATA SCIENCE. ANNALS OF INFORMATION SYSTEMS, pp. 163-178. Springer, Cham (2018).

16. Murphy, K.: Naive Bayes classifiers. In: Lectures, pp. 1-5. University of British Columbia, Vancouver (2006).

17. Breiman, L.: Random forests. Mach. Learn. 45(1), 5-32 (2001). https://doi.org/10.1023/A:1010933404324 .

18. Zhang, Z.: Artificial neural network. In: Zhang, Z. (ed.) Multivariate Time Series Analysis in Climate and Environmental Research, pp. 1-35. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-67340-0.

19. Bashar, M.A., Nayak, R., Suzor, N., Weir, B. Misogynistic Tweet Detection: Modelling CNN with Small Datasets. In: Islam R. et al. (eds) DATA MINING. AUSDM 2018. Communications in Computer and Information Science, vol 996, pp 3-16. Springer, Singapore (2019).

Topic modeling oftext content for monitoring the employee's efficiency via his Internet activity

Artem Beresnev, Sergey Dudorov, Natalia Gusarova, Natalia Dobrenko, Alexandra Vatyan,

ITMO University Saint-Petersburg, Russia

[e-mail], [e-mail], [e-mail], [e-mail], [e-mail],

Niyaz Nigmatullin, Nikolay Vedernikov, Artem Vasilev, Anatoly Shalyto ITMO University Saint-Petersburg, Russia

[e-mail], [e-mail], [e-mail], [e-mail]

Abstract — In this paper, we investigate the possibilities of topic modeling techniques, namely PLSA, LDA and ARTM, for monitoring the employee's efficiency via his Internet activity. We have created a set of scenarios for representation of the employee's behavior in information retrieval, and we have formed a set of datasets for investigating the results of the information retrieval if the employees fulfil it with different efficiency. We have shown experimentally that in our task the ARTM model provides the best quality of topic extraction in comparison with PLSA and LDA models. At last, we have experimentally selected metrics of quality of the ARTM model which allow to keep track of the employee effectiveness in real information search. The results of our work can be used for creating additional KPI of the employee, for creating the corporative recommendatory systems and the corporative corpus of informational and educational materials, for the creation of models of influence on social networks in the conditions of outer target intervention.

The work was financially supported by the Government of Russian Federation (Grant 074-U01).

I. Introduction

Maintenance of the necessary level of employee's efficiency is one of the main tasks of business. In general, it can be measured in two ways: as the amount of units of a product or service that an employee handles in a defined time frame (productivity) or as a ratio of an employee's actual time to perform each unit of service against the theoretical time needed to complete it (actual efficiency). But it is difficult to apply such approach to employees whose activities are connected to information search, in particular, with Internet browsing. For example, if the company is engaged in the development of computer games, then employees shall trace new decisions of competitors. In this case it is useless or even harmful to list the websites allowed or forbidden to visit, because it is unknown, where exactly these decisions will appear. Any employee selects his own strategy of information search; but the effective employee's strategy leads to the success of the whole business, and vice versa.

Hence, in order to make a distinction between useful (effective) and useless (ineffective) employees' web-browsing behavior the manager shall compare the behavior of each employee to a reference model of the effective employee.

Modeling of web-user behavior is used in various application-oriented tasks, including web sites adaptation, web personalization, recommendation systems, etc. In the majority of applications revealing the typical behavior of the user or a user group is required. The plethora of solutions is already proposed for this purpose (see, for example, [1-5]). However, in case of modeling of employee's web-behavior the task is set differently: we need to build an actual reference model of the effective employee behavior and at the same time we need to trace deviations of the behavior of other employees from a reference model. Here arise a number of problems which are a subject of the active researches today [3, 6, 7].

In this paper, one of the aspects of these problems is experimentally investigated, namely, possibilities of topic modeling techniques for monitoring the employee's efficiency via his Internet activity. Specifically, we investigate the influence of topic models regularization on the discernability of effective and ineffective employees' web-browsing behavior.

The article is organized as follows. In Section 2, related studies are presented and the research tasks formulated. Section 3 describes the research methodology, in Section 4 are the results of the experiments and their discussion. Section 5 presents the conclusion and a direction of further work.

II. RELATED work

Despite already mentioned diversity of the proposed application-oriented solutions, the amount of types of representative variables describing the web user browsing behavior is small. In [5, 8] three types of such variables are selected. These are the web structure, the web user session and the web content.

In the first case the web resource is considered as the directed graph consisting of a collection of nodes containing content information and of vertexes representing hyperlinks and intra-links. This representation allows evaluating all transitions between

nodes, which is useful for stochastic process descriptions. However, such approach allows to consider only static structure of web resources, and it is a serious shortcoming for modeling of user behavior in the modern high-dynamic Internet. Indeed, we did not manage to find in available sources any example of applications based on this approach in pure form.

In the second case the behavior of the user of a web resource is represented as a browsing trajectory, which is a sequence of his relocation among nodes taking into account time spent on each node. This approach is much richer, and a set of technical solutions is suggested on its basis.

For example, the authors [1] consider the summary time of a session (named session length) as the single parameter and build its statistical distribution. They show that for different types of the websites (namely commercial and academic) the type of distribution is various. The authors argue that their approach helps to identify the nature of the users and to divide them into several different groups. However, judging by the experimental data provided in the paper, one can obtain here only too aggregated partitions.

In [2] authors also analyze time spent by the user on a resource. They use a clustering approach to make groups of similar Web pages by distributions of spent times. If pages are in the same type, this distribution can show the best ones. So, the authors consider the time spent on certain web page as an indicator of quality of its content. It seems that this approach can be expanded to assessment of quality of the whole web user session, but in the explicit form it is not represented.

In [3] authors present the method of web log analysis based on exploring the rules of web log records. They argue that it improves the quality of identifying the potential customers of the website. In order to further improve the quality of user behavior analysis, they explore the differences in user's browsing behavior based on different types of access events, separately selecting the analytic events.

More complex processing of web logs is fulfilled in [5]. The authors use pattern analysis along with association rules in order to elicit log structures common to certain users' groups.

In [9] authors consider user's behavior in web-browsing as a so called interest sequence. Namely, they take into account the rating given by the each user on the certain item at the certain timestamp. The length of the longest common sub-sequence along with the count of all common sub-sequences are defined as the measure of similarity of the user's behavior.

In [10] authors use proxy log for user profiling. They apply Latent Dirichlet allocation (LDA) [11] model to the user's URL sessions. Thereby the set of the latent URL-topics is created in which the profile of the specific user is built. An improved variant of this idea is presented in [12].

Nevertheless, judging by the literature, the most extensive opportunities for modeling of web-users behavior arise when using the web content as representative variables or when combining this approach with others mentioned above.

For example, in [6] the authors describe changes in the behavior of the user through the tracing of its personal profile in Twitter. The personal profile is being built for each user and consists of graphs with edges of two types - follows (user, user j) and tweets about (user, organization, hashtag). The first one describes the position of each user in the whole network, and the second one - the deviation of his interests among other users. Authors highlight that the offered method helps to trace changes in employees' behavior and thus to improve their efficiency.

In [4] authors use neural word embedding for modeling web-user behavior. In order to take into account the current changes in users' behavior, authors suggest giving more weights to those concepts which are associated with positive evaluation.

Today it is conventional that the full-scale usage of web content for simulation of the behavior of the user should be based on the analysis of the semantic organization of relevant domain. To this end, different knowledge bases can be used, including taxonomies, flat databases and ontologies (see literature review within [13]).

For example, in [13] the textual contents of all the web-pages which the user has visited during browsing sessions is being fetched, preprocessed and mapped to appropriate concepts (or classes) in the reference ontology according to the cosine similarity algorithm. In order to adapt the mapping process to dynamic changes in user's behavior, the authors introduce multi-agent system classifying concepts depending on the frequency of their appearance in the long-term and short-term, thus forming the user's session-based profile. The authors argue, that their system allows to supervise slight changes in behavior of users both in informative, and in dynamic aspect.

As a rule, application of ontological approach requires heavy methodologies and leads to sophisticated technical solutions. To this end, the decisions based on the latent factor revealing are of great interest in modeling user behavior [7, 14]. Here, in turn, the approaches connected to topic modeling are estimated as very perspective (see reviews [15, 16]).

To the best of our knowledge, the work [17] was one of the first to suggest such approach for modeling employee's behavior; however, other decisions mentioned below can also be easily transformed for this purpose. For example, authors [18] offer rather simple way to find user search preferences from web browsing histories. They store the browsing history of various users in the form of a CSV file and then process it using LDA for topic modeling. In order to reveal the dynamics of users' interests, they

visualize the LDA results for each selected topic in the form of word vector and of the word cloud. However, visualization allows receiving the rather conditional and integrated assessment.