Gale M. Lucas PhD
Professor of Research
HRA 412E
.(JavaScript must be enabled to view this email address)
Website: chan.usc.edu/tel
Dr. Gale M. Lucas is a leading researcher in human-computer interaction and trust-in-automation in the workplace. Specifically, her expertise is in teaming, training, and co-working with Artificial Intelligence (AI). After obtaining a PhD from Northwestern University, she completed her postdoc on the topic of AI at USC’s Institute for Creative Technologies (ICT), and then went on to serve as the Director of ICT’s Technology Evaluation Lab. From there, she joined the Chan Division, also serving as a member of Chan’s Statistics Information Graphics Management & Analysis (SIGMA) Core.
Postdoctoral Fellowship
in Computer Science
2015 | University of Southern California
Doctor of Philosophy (PhD)
in Psychology
2010 | Northwestern University
Master of Arts (MA)
in Psychology
2008 | Northwestern University
Bachelor of Arts (BA)
in Psychology
2005 | Willamette University
Wang, Z., Rodrigues, P. B., Fang, Y., Rathi, V., Lucas, G. M., Roll, S. C., Becerik-Gerber, B., & Soibelman, L. (2027). Characterizing operators’ perceived information needs in teleoperated demolition across task contexts. Advanced Engineering Informatics, 77, 105456. https://doi.org/10.1016/j.aei.2026.105456 Show abstract
Teleoperated robots can help address skilled equipment operator shortages and improve operator safety in demolition, but physical separation from cluttered sites creates challenges in information perception. Operators depend on interface-mediated information to understand site conditions and control robots, yet limited empirical evidence shows how they assess information needs as site conditions vary and task demands change. This paper investigates how operators’ perceived information needs vary across task contexts in teleoperated demolition and explores potential differences associated with operator characteristics. Forty-six construction workers completed a scenario-based evaluation after real-world teleoperation exposure. The evaluation included six textual demolition scenarios representing task contexts defined by three site conditions and two task types: robot navigation and arm manipulation. For each scenario, participants evaluated interface features provided through a teleoperation workstation, with perceived information needs captured by rating feature usefulness, selecting necessary features, and ranking feature priorities. Results show context-level selective reweighting in perceived information needs across the evaluated task contexts. When multiple site constraints coexisted, operators’ usefulness ratings changed selectively across the potentially relevant information sources, rather than increasing uniformly across all these sources. Usefulness ratings also differed by task type, with these task-related needs remaining evident across the tested conditions. These findings provide bounded empirical evidence of context- level selective reweighting in perceived information needs. The design implications inform future adaptive interfaces and provide a conceptual basis for context-to-information mapping, along with a preliminary qualitative reconfiguration criterion and several indicators.
Seyedrezaei, M., Becerik-Gerber, B., Awada, M., Lucas, G., & Roll, S. C. (2026). Interaction effects of indoor environmental quality on stress among young adults: An experimental study. Indoor and Built Environment, 35(10), 1888-1914. https://doi.org/10.1177/1420326X261480440 Show abstract
Background. Indoor environmental quality (IEQ) significantly influences occupant stress in office settings, yet the combined effects of multiple environmental stressors, moderated by personal characteristics, remain underexplored.
Objective. This study examines the interaction effects of air temperature, lighting correlated colour temperature (CCT), and background noise on stress responses among office workers, with attention to individual differences in gender and body mass index (BMI).
Methods. A mixed-design controlled experiment was conducted with 52 young adults in an open plan office within a North American Mediterranean climate, varying temperature (20°C vs. 25°C), CCT (2700 K vs. 6500 K), and noise level (50 dB vs. 65 dB). Stress was assessed using physiological and psychological indicators.
Key Results. Stress responses were shaped not by individual environmental attributes in isolation, but through complex interactions among temperature, CCT, and noise, further moderated by BMI and gender. Individuals with overweight/obese BMI showed heightened sensitivity to warm temperatures, cool CCT, and high noise, while males exhibited greater sympathetic activation under warm and noisy conditions compared to females.
Implications. These findings underscore that a one-size-fits-all approach to office environmental design is insufficient. Accounting for occupant heterogeneity, particularly BMI and gender, is essential for developing personalized IEQ strategies that promote workplace well-being.
Keywords. health, heart rate variability, indoor environmental quality, interaction effect, stress
Wang, Z., Rodrigues, P. B., Fang, Y., Soibelman, L., Becerik-Gerber, B., Lucas, G. M., & Roll, S. C. (2026). Design of a sensory-enhanced workstation prototype with a multimodal interface for teleoperated demolition. In Construction Research Congress 2026 (pp. 130-140). American Society of Civil Engineers. https://doi.org/10.1061/9780784486979.013 Show abstract
Demolition is among the most complex and hazardous sectors of construction, where workers need to work in rapidly changing and cluttered environments. Teleoperation offers a promising approach to reducing operators’ exposure to on-site hazards by enabling remote control of demolition robots, yet existing interfaces are often insufficiently grounded in operators’ needs or validated through real-world testing. This paper presents a sensory-enhanced teleoperation workstation prototype that operationalizes operators’ sensory needs identified in prior studies. The prototype translates these needs into multimodal sensing channels, associated data-flow structures, and corresponding workstation features. It provides a platform for subsequent studies to understand how operators engage with sensory information in real-world scenarios. The prototype also holds the potential to inform operator training and teleoperation research for other demolition machines and related contexts.
Seyedrezaei, M., Becerik-Gerber, B., Roll, S. C., Lucas, G., & Narayanan, S. (2026). Longitudinal exploration of productivity and well-being in hybrid work through ecological momentary assessment and environmental sensing. Journal of Occupational & Environmental Medicine. Advance online publication. https://doi.org/10.1097/JOM.0000000000003833 Show abstract
Objective. To explore how hybrid workspaces (home vs. office) affect productivity and well-being, and whether indoor environmental quality (IEQ) mediates these associations.
Methods. Nineteen office workers were followed for four months, contributing 4,559 ecological momentary assessments (EMAs) of productivity, mood, stress, and eustress-distress appraisal. Continuous IEQ monitoring captured CO₂, TVOCs, PM₂.₅, temperature, humidity, and noise. Mixed-effects and multilevel mediation models assessed within-person associations.
Results. Office work was associated with higher productivity and greater eustress-distress coexistence, while home work was associated with higher boredom. Mood differences were small, and stress was modestly lower at home. Home workspaces showed poorer air quality. CO₂ and TVOCs were indirectly associated with mood and stress differences.
Conclusions. Hybrid work outcomes differ by location and are partly explained by IEQ, suggesting residential air quality as an occupational health intervention target.
Han, Z., Becerik-Gerber, B., Liu, R., Lucas, G. M., Doolab, Y., Zein, S., Atkins, M., Blass, D., & Miles, L. (2026). Emergency dynamics modeling with an agent-based framework: Active shooter scenario case study. International Journal of Disaster Risk Reduction, 142, 106226. https://doi.org/10.1016/j.ijdrr.2026.106226 Show abstract
Modeling human-induced emergencies requires a structured representation of interactions among occupants, hazards, and built environments. While existing simulation approaches provide important insights, behavioral processes, hazard progression, and architectural conditions are often examined separately. This study develops a parameterized agent-based modeling framework that integrates heterogeneous occupant behaviors, hazard dynamics, and environmental configurations within a commercially available crowd-simulation platform to extend its usability within the academic and professional communities. The framework comprises environmental modeling, agent definition, interaction logic, risk evaluation, and simulation execution, enabling systematic configuration of behavioral rules and scenario parameters through customized scripting. An Active Shooter Incident (ASI) in a school building is simulated as a demonstrative case study to illustrate the framework's capability to configure and test emergency scenarios under different behavioral, hazard and environmental assumptions. Using the framework, the study examines the effects of classroom occupancy, door width, and door-locking mechanisms on occupant safety, evaluated using a token-based high-risk metric and factorial statistical analysis. Results indicate that higher occupancy increases exposure risk, timely door locking substantially improves safety outcomes, and the influence of door width is context-dependent. The consistency of these patterns with prior research supports the behavioral plausibility of the modeling approach. Overall, the study contributes a practical and extensible framework for operationalizing human–hazard–environment interactions in similar agent-based simulations and for generating scenario-specific insights to support design and preparedness strategies in human-induced emergency contexts.
Keywords. Agent-based modeling, Behavior simulation, Active shooter incidents, Human-hazard-environment interaction, Human-induced emergency
Rodrigues, P. B., Wang, Z., Fang, Y., Lucas, G., Roll, S., Becerik-Gerber, B., & Soibelman, L. (2026). Teleoperation interfaces for compact demolition robots: Results of a pilot study with construction workers. In Proceedings of the European Conference on Computing in Construction. Europen Council for Computing in Construction. https://doi.org/10.35490/EC3.2026.422 Show abstract
As teleoperated robots become more common on construction sites, human-robot interfaces must better support operators’ sensory needs. This paper presents results from a pilot study comparing a proposed sensory-enhanced teleoperation interface for compact demolition robots with a standard teleoperation interface. Recommendations from demolition stakeholders informed the design of the sensory-enhanced interface, while the standard interface was based on commercially available teleoperated construction equipment. The goal of this study was to refine the sensory-enhanced interface based on user feedback. Ten construction workers tested the interfaces, and we measured their perceived usability, telepresence, and trust in technology.
Seyedrezaei, M., Becerik-Gerber, B., Lucas, G., Roll, S. C., & Narayanan, S. (2026). Environmental comfort in hybrid work: A longitudinal preliminary field study of home and office workspaces. In Proceedings of Indoor Air 2026. International Society of Indoor Air Quality and Climate. Full text Show abstract
This study examines environmental comfort differences between home and office workspaces and their associations with productivity and well-being among 19 knowledge workers monitored over four months via ecological momentary assessments. Participants rated comfort across four domains (lighting, temperature, air quality, noise) and reported productivity, mood, stress, and stress states four times daily. Mixed-effects models indicated domain-specific patterns: offices showed higher comfort for lighting (OR=0.56) and noise (OR=0.48), while temperature showed individual heterogeneity. Lighting and temperature comfort were positively associated with productivity (4.3% and 2.4% increase per comfort level), while comfort showed weak associations with mood and stress. Bayesian multinomial models suggested that higher comfort was associated with lower odds of boredom (ORs=0.40-0.56). Substantial between-person heterogeneity was observed in the direction and magnitude of home-office differences in comfort, well-being, and productivity, suggesting that hybrid policies may not be one-size-fits-all and may benefit from greater flexibility, though these findings should be interpreted as exploratory.
Zang, T., Awada, M., Becerik-Gerber, B., Roll, S. C., Lucas, G., & Narayanan, S. (2026). Does your office make you feel worse? Linking environmental exposure and temporal factors to cognitive symptoms. Indoor Environments, 3(2), 100171. https://doi.org/10.1016/j.indenv.2026.100171 Show abstract
Symptoms such as fatigue, headache, low motivation, and difficulty concentrating affect office workers’ cognitive and overall well-being. This study investigates the dynamic relationships among temporal patterns, indoor environmental quality (IEQ) factors, and cognitive symptoms in real offices. We collected about four months of continuous environmental data and ecological momentary assessments (EMAs) from ten office workers and integrated XGBoost and SHAP analysis to identify those relationships. Model performance varied by symptom and participant. Under random train-test splitting, accuracy generally exceeded 70% with F1 scores ranging from 40% to 80%. Performance declined under a temporal split, highlighting the real-world challenge of predicting future symptom states from past observations. Based on SHAP analysis, hour of day and day of week emerged as significant predictors, revealing temporal patterns in cognitive symptoms. Within observed one-hour windows prior to reported symptoms, higher relative humidity was generally associated with increased risk of cognitive symptoms, while higher illuminance was associated with reduced symptom risk. Associations with sound pressure level and TVOCs varied across participants, with no consistent directions. Our data-driven approach provides insights into temporal factors and environmental exposures associated with cognitive symptoms using naturalistic data. These findings highlight the combined influence of temporal and IEQ factors on cognitive symptoms and contribute to the development of adaptive, user-centric workplaces that support cognitive well-being management.
Keywords. Office worker, Cognitive symptoms, Well-being, Machine learning, Longitudinal data, Temporal patterns, Indoor environmental quality
Kwon, D., Shrestha, K., Han, B., Lin, S., Hale, J., Gratch, J., Mataric, M., & Lucas, G. M. (2026). Can LLMs truly embody human personality? Analyzing AI and human behavior alignment in dispute resolution. Proceedings of the AAAI Conference on Artificial Intelligence, 40(45), 38790–38798. https://doi.org/10.1609/aaai.v40i45.41223 Show abstract
Large language models (LLMs) are increasingly used to simulate human behavior in social settings such as legal mediation, negotiation, and dispute resolution. However, it remains unclear whether these simulations reproduce the personality–behavior patterns observed in humans. Human personality, for instance, shapes how individuals navigate social interactions, including strategic choices and behaviors in emotionally charged interactions. This raises the question: Can LLMs, when prompted with personality traits, reproduce personality-driven differences in human conflict behavior? To explore this, we introduce an evaluation framework that enables direct comparison of human-human and LLM-LLM behaviors in dispute resolution dialogues with respect to Big Five Inventory (BFI) personality traits. This framework provides a set of interpretable metrics related to strategic behavior and conflict outcomes. We additionally contribute a novel dataset creation methodology for LLM dispute resolution dialogues with matched scenarios and personality traits with respect to human conversations. Finally, we demonstrate the use of our evaluation framework with three contemporary closed-source LLMs and show significant divergences in how personality manifests in conflict across different LLMs compared to human data, challenging the assumption that personality-prompted agents can serve as reliable behavioral proxies in socially impactful applications. Our work highlights the need for psychological grounding and validation in AI simulations before real-world use.
Liu, R., Becerik-Gerber, B., & Lucas, G. M. (2026). Technology acceptance in high-stakes serious games: Extending Technology Acceptance Model (TAM) with affective and experiential predictors in active shooter preparedness. International Journal of Human–Computer Interaction. Advance online publication. https://doi.org/10.1080/10447318.2026.2635682 Show abstract
Active shooter incidents (ASIs) represent extreme threats to public safety. Virtual reality (VR) serious games offer promising training potential, but user acceptance in high-stakes and emotionally intense contexts is underexplored. This study extends the Technology Acceptance Model (TAM) by incorporating affective (perceived enjoyment) and experiential (self-reported engagement) factors. Using a desktop-based VR ASI preparedness simulation with 50 healthcare workers, we tested the extended TAM model. Core TAM relationships hold for ease of use and usefulness, but enjoyment and engagement play distinct roles. Enjoyment strongly predicted adoption, fully mediating the effect of ease of use and partially mediating usefulness. It was associated with high-arousal positive emotions (e.g., inspiration, pride, enthusiasm), reflecting eudaimonic states rather than hedonic fun. Engagement also significantly promoted adoption, though with weaker mediation effects. These findings extend TAM to high-stakes serious games and suggest that effective design should cultivate meaningful enjoyment and engagement without undermining scenario seriousness.
Keywords. Technology acceptance model, perceived enjoyment, engagement, serious games, virtual reality
Charnsethikul, P., Fattepurkar, A., Desai, D., Lucas, G. M., & Mirkovic, J. (2026). Replication: A study on how users (don’t) use password managers. In Proceedings of the 2026 Symposium on Usable Security and Privacy (pp. 1–18). Internet Society. https://doi.org/10.14722/usec.2026.23032 Show abstract
We replicated the study by Mayer et al. [1] on password habits and password manager (PM) usage at a large private US university. We conducted an online survey (n = 437) and found high awareness (96%) and usage (94%) of PMs, but limited use of password generation (26%) and substantial password reuse, with participants reusing more than half of their passwords. These findings are consistent with the original study. However, we found that participants were unlikely to adopt a free third-party PM offered by the university, contrary to the original findings. Extending the original study, we found that awareness of the free PM was low: only 35% knew about it, and its adoption was even lower, at just 15%. We also found that faculty had the strongest password habits, while students had the weakest. Based on our findings, we provide recommendations for increasing the use of password generation features, broadening adoption of an institution-provided PM, and guiding future replication efforts.
Seyedrezaei, M., Reyes, M., Becerik-Gerber, B., Roll, S. C., Lucas, G., & Narayanan, S. (2026). Longitudinal analysis of IEQ variability in office and home workspaces: Insights for the future of hybrid work. In A. Jafari, & Y. Zhu (Eds.), Computing in Civil Engineering 2025: Computational and Intelligent Technologies (pp. 1058-1067). American Society of Civil Engineers. https://doi.org/10.1061/9780784486436.113 Show abstract
As hybrid work gains traction, understanding exposure to varying indoor environmental quality (IEQ) attributes becomes increasingly important for promoting health, productivity, and equity in workplace models. This longitudinal study investigated key IEQ attributes—temperature, relative humidity (RH), carbon dioxide (CO₂), particulate matter (PM₂.₅), total volatile organic compounds (TVOC), and ambient noise—across 12 participants’ home and office workspaces. Data were collected over 4 months through continuous monitoring and ecological momentary assessments (EMAs). Results revealed significant differences in IEQ variability between the two workspace types. Office workspaces generally maintained more stable conditions due to advanced climate control systems, while home workspaces exhibited greater fluctuations likely influenced by weather, building design, and occupant behaviors. Notably, TVOC and PM₂.₅ levels occasionally exceeded recommended thresholds in some home workspaces, underscoring potential health risks. These findings highlight the need for adaptive strategies, such as improved ventilation, air purification, and occupant education, to enhance IEQ in hybrid workspaces.
Tags. Building design, Control systems, Health hazards, Human and behavioral factors, Hybrid methods, Volatile organic compounds, Work zones
Parga, M., Roll, S., & Lucas, G. (2026). The value of occupational justice for socially responsible technology: An interdisciplinary exploration with the field of human-computer interaction. Brazilian Journal of Occupational Therapy, 34(Special Issue), 1002. Full text Show abstract
Introduction. Applications of advanced technology have vast implications for everyday occupations when considering how interacting with technology can influence the nature and quality of daily engagement. With examples including artificial intelligence, assistive devices, and modified built environments, technological advancements pose new considerations for occupational therapists and scientists to apply the framework of occupational justice (FOJ).
Objectives. The presentation will explore how occupational therapy and occupational science (OT/OS) knowledge is pertinent for informing positive engagement between technologies and occupational beings.
Approach. I will draw from interdisciplinary connections between OT/OS and the field of human-computer interaction (HCI), which is primarily concerned with human engagement with technology, to create space for occupational scientists and OTs to consider how an occupational lens is directly relevant to informing socially responsible technologies that promote inclusivity.
Practice Implications. Advanced technologies have become an integrated component of daily life for many. These technologies can potentially be leveraged to enhance OT practice and enrich daily occupation; however, it is important to explore concerns that arise with new technological applications. By leveraging the FOJ, OTs and occupational scientists can be involved in ongoing discussion regarding the impact of technology on everyday living and the development of socially responsible technologies.
Conclusion. The fields of OT/OS present unique and useful perspectives for informing socially responsible design and applications of technology. Occupational justice has interdisciplinary relevance to other fields such as HCI and occupational researchers and professionals can continue to understand and perpetuate the valuable implications for considering humans as occupational beings and not solely “users” in technology interactions.
Fang, Y., Aldrich, R., Lucas, G., Roll, S., Becerik-Gerber, B., & Pyatak, B. (2026). The evolving nature of work in the digital age: Exploring construction workers’ occupational adaptations to technological transformations. Brazilian Journal of Occupational Therapy, 34(Special Issue), 1922. Full text Show abstract
Objectives. The rapid advancement of technological innovations is driving significant changes in work. This dissertation explores the occupational adaptation process among construction workers transitioning from manual labor to technology-mediated work (e.g., robotics, automation, artificial intelligence). The specific aims include: 1) identify critical constructs of occupational adaptation and worker health related to workplace technology transformations, and 2) characterize the relationship between occupational adaptation and worker health and well-being.
Methods. This mixed-methods study uses an exploratory sequential design, where a series of qualitative interviews (Aim 1) inform the development and administration of a quantitative survey (Aim 2). We have completed six narrative interviews and three focus groups with construction operators and company supervisors to understand work contexts and technological adoption. These findings informed additional in-depth, semi-structured interviews with workers who have direct experiences with construction innovations. Based on the findings of Aim 1, a quantitative survey explores the relationship between constructs relevant to occupational adaptation and specific domains of worker health and well-being.
Impact/Results. Preliminary Aim 1 data suggest that robotics with remote-operation capacities remove workers from worksite hazards and reduce manual labor, mitigating physical fatigue after long workdays. Increased efficiency and reduced time pressure also improve mental health and job satisfaction. Regarding occupational adaptation, participants who started as fieldwork laborers upskilled and transitioned into project engineer and manager roles as they adapted to collaborating with advanced technologies on job sites.
Conclusion/Contribution to the Occupational Therapy Profession. As work evolves rapidly, occupational therapy professionals must expand our knowledge of technology-mediated occupations. These findings will inform the design of human-centered workplace technology that promotes worker well-being.
Liu, R., Becerik-Gerber, B., & Lucas, G. M. (2025). Investigating the role of personal factors in shaping responses to active shooter incidents using machine learning. In Computing in Civil Engineering 2024 (pp. 341–350). Reston, VA: American Society of Civil Engineers. https://doi.org/10.1061/9780784486115.036 Show abstract
This study bridges the knowledge gap on how personal factors affect building occupants’ responses in active shooter situations by applying interpretable machine learning methods to data from 107 participants. The personal factors studied are training methods, prior training experience, sense of direction, and gender. The response performance measurements consist of decisions (run, hide, multiple), vulnerability (corresponding to the time a participant is visible to a shooter), and pre-evacuation time. The results indicate that propensity to run significantly determines overall response strategies, overshadowing vulnerability, and pre-evacuation time. The training method is a critical factor where VR-based training leads to better responses than video-based training. A better sense of direction and previous training experience are correlated with a greater propensity to run and less vulnerability. Gender slightly influences decisions and vulnerability but significantly impacts pre-evacuation time, with females evacuating slower, potentially due to higher risk perception. This study underscores the importance of personal factors in shaping responses to active shooter incidents.
Kwon, D., Hae, J., Clift, E., Shamsoddini, D., Gratch, J., & Lucas, G. M. (2025). ASTRA: A negotiation agent with adaptive and strategic reasoning via tool-integrated action for dynamic offer optimization. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (pp. 16217–16238). Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.emnlp-main.821 Show abstract
Negotiation requires dynamically balancing self-interest and cooperation within the flow of conversation to maximize one’s own utility. Yet, existing agents struggle due to bounded rationality in human data, low adaptability to counterpart behavior, and limited strategic reasoning. To address this, we introduce principle-driven negotiation agents, powered by ASTRA, a novel framework for turn-level offer optimization grounded in two core principles: opponent modeling and Tit-for-Tat reciprocity. ASTRA operates in three stages: (1) interpreting counterpart behavior, (2) optimizing counteroffers via a tool-integrated action with a linear programming (LP) solver, and (3) selecting offers based on strategy assessment and the partner’s acceptance probability. Through simulations and human evaluations, our agent effectively adapts to an opponent’s shifting stance and achieves favorable outcomes through enhanced adaptability and strategic reasoning. Beyond enhancing negotiation performance, it also serves as a powerful coaching tool, offering interpretable strategic feedback and optimal offer recommendations beyond human bounded rationality, with its potential further validated through human evaluation.
Kwon, D., Shrestha, K., Han, B., Lee, E. H., & Lucas, G. M. (2025). Evaluating behavioral alignment in conflict dialogue: A multi-dimensional comparison of LLM agents and humans. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (pp. 16366–16380). Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.emnlp-main.828 Show abstract
Large Language Models (LLMs) are increasingly deployed in socially complex, interaction-driven tasks, yet their ability to mirror human behavior in emotionally and strategically complex contexts remains underexplored. This study assesses the behavioral alignment of personality-prompted LLMs in adversarial dispute resolution by simulating multi-turn conflict dialogues that incorporate negotiation. Each LLM is guided by a matched Five-Factor personality profile to control for individual variation and enhance realism. We evaluate alignment across three dimensions: linguistic style, emotional expression (e.g., anger dynamics), and strategic behavior. GPT-4.1 achieves the closest alignment with humans in linguistic style and emotional dynamics, while Claude-3.7-Sonnet best reflects strategic behavior. Nonetheless, substantial alignment gaps persist. Our findings establish a benchmark for alignment between LLMs and humans in socially complex interactions, underscoring both the promise and the limitations of personality conditioning in dialogue modeling.
Wang, Y., Lucas, G. M., Becerik-Gerber, B., & Ustun, V. (2025). Implicit behavioral alignment of language agents in high-stakes crowd simulations. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (pp. 30681–30698). Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.emnlp-main.1562 Show abstract
Language-driven generative agents have enabled large-scale social simulations with transformative uses, from interpersonal training to aiding global policy-making. However, recent studies indicate that generative agent behaviors often deviate from expert expectations and real-world data—a phenomenon we term the Behavior-Realism Gap. To address this, we introduce a theoretical framework called Persona-Environment Behavioral Alignment (PEBA), formulated as a distribution matching problem grounded in Lewin’s behavior equation stating that behavior is a function of the person and their environment. Leveraging PEBA, we propose PersonaEvolve (PEvo), an LLM-based optimization algorithm that iteratively refines agent personas, implicitly aligning their collective behaviors with realistic expert benchmarks within a specified environmental context. We validate PEvo in an active shooter incident simulation we developed, achieving an 84% average reduction in distributional divergence compared to no steering and a 34% improvement over explicit instruction baselines. Results also show PEvo-refined personas generalize to novel, related simulation scenarios. Our method greatly enhances behavioral realism and reliability in high-stakes social simulations. More broadly, the PEBA-PEvo framework provides a principled approach to developing trustworthy LLM-driven social simulations.
Han, B., Hoegen, J., Lei, S., Lucas, G. M., Shore, D., Parkinson, B., & Gratch, J. (2025). How expression, context and perspective determine judgments of emotion. IEEE Transactions on Affective Computing. Advance online publication. https://doi.org/10.1109/TAFFC.2025.3618588 Show abstract
Within the field of Affective Computing, facial expressions were historically assigned emotion labels by annotators without knowledge of the context in which the expressions were produced. Often these 3rd-person impressions were used instead of 1st-person judgments about the target's feelings. The field now appreciates that 1st- and 3rd-person judgments can differ dramatically and that context plays a central role in explaining these differences. More recently, there is growing appreciation of the importance of a 2nd-person perspective. When people are engaged in a social task, impressions of their partner's emotions differ from the impressions of detached bystanders. In this paper, we explore how facial expressions, contexts and perspective (1st, 2nd-, and 3rd-person) interact to determine judgments of emotion. We explore this using automatic facial expression analysis of natural expressions produced in the context of a social task (the prisoner's dilemma). Our findings suggest that expression and context combine to determine judgments, but the way they combine differs depending on perspective. We discuss the implication of these findings for automatic emotion recognition methods and the inferences such methods can support.
Keywords. context, facial expression, perspective, valence, prisoner's dilemma
Charnsethikul, P., Zunquti, A., Lucas, G. M., & Mirkovic, J. (2025). Navigating social media privacy: Awareness, preferences, and discoverability. Proceedings on Privacy Enhancing Technologies, 2025(4), 620–638. https://doi.org/10.56553/popets-2025-0148 Show abstract
Social media platforms provide various privacy settings, which users can adjust to fit their privacy needs. Platforms claim that this is sufficient – users have power to accept the default settings they like, and change those they do not like. In this paper, we seek to quantify user awareness of, preferences around and ability to adjust social media privacy settings. We conduct an online survey of 541 participants across six different social media platforms: Facebook, Instagram, X, LinkedIn, TikTok, and Snapchat. We focus on nine privacy settings that are commonly available across these platforms, and evaluate participants’ preferences for privacy, awareness of the privacy settings and ability to locate them. We find that default settings are ill-aligned with user preferences – 92% of participants prefer at least one of the privacy options to be more private than the default. We further find that users are generally not aware of privacy settings, and struggle to find them. 80% of participants have never seen at least one privacy setting, and 79% of participants rated at least one setting as hard to find. We also find that the fewer privacy settings a user has seen, the harder for them to locate those settings, and the higher the level of privacy they desire. Additionally, we find that there are significant differences in privacy setting preferences and usability across different user age groups and across platforms. Older users are more conservative about their privacy, they have seen significantly fewer privacy settings, and they spend significantly more time locating them than younger users. On some platforms, like LinkedIn, users opt for higher visibility, while on others they prefer more privacy. Some platforms, like TikTok, make it significantly easier for users to locate privacy settings. Based on our findings, we provide recommendations on default values and how to improve usability of privacy settings on social media.
Keywords. Privacy, Social Media, Awareness, Default Privacy Settings
Rodrigues, P. B., Lucas, G. M., Fang, Y., Wang, Z., Roll, S. C., Becerik-Gerber, B., & Soibelman, L. (2025). Understanding operators’ sensory needs for human–robot interaction in teleoperated demolition. Journal of Construction Engineering and Management, 151(12), 04025207. https://doi.org/10.1061/jcemd4.coeng-16911 Show abstract
Despite the benefits of remotely operated compact demolition machines in constrained and hazardous environments, many safety and occupational risks remain. Operators often stay close to the machines, exposing themselves to the risks of structural collapses, falling debris, dust, and noise. In this study, we employed a generative user experience (UX) framework with six demolition industry professionals to inform the design considerations for a sensory-enhanced teleoperation interface for demolition robots. Through narrative interviews and focus groups, we identified demolition operators’ sensory needs during traditional demolition tasks involving manned and remotely operated machines and explored potential sensory features for a teleoperation interface for demolition robots. To increase the trustworthiness of the findings, we subsequently conducted a verification strategy where over 60 demolition stakeholders tested and provided feedback on the proposed teleoperation interface features within a virtual environment. Findings indicate that demolition operators use multiple sensory channels to perceive and interact with the environment during traditional demolition tasks. However, certain site conditions (e.g., noise, constrained spaces, clutter) still compromise their performance, safety, and well-being. Multimodal, simple, safe, and personalizable teleoperation interfaces that provide appropriate levels of control over the robots are preferred. Key suggestions included improvements in the robot’s controllers and the inclusion of mechanisms to facilitate communications with on-site workers and the perception of hazards. The findings inform the design considerations for a sensory-enhanced teleoperation interface for compact demolition robots. Additionally, the proposed generative UX framework can be applied to other construction applications to facilitate identifying operators’ needs and designing teleoperation interfaces
Liu, R., Wu, W., Becerik-Gerber, B., Lucas, G. M., Laboy, M., & Fannon, D. (2025). Reinforcement learning for evaluating school safety designs in active shooter incidents. Advanced Engineering Informatics, 67, 103575. https://doi.org/10.1016/j.aei.2025.103575 Show abstract
School shootings pose significant threats to public safety, requiring innovative approaches to evaluate and improve building design for emergency preparedness. This study presents a reinforcement learning (RL)-based simulation framework to assess the impact of school layouts on active shooter incident (ASI) outcomes. Using four generalized school prototypes, the RL-based model simulates adaptive shooter behavior, capturing dynamic interactions between building design, occupant movement, and threat progression. The study systematically examines how exit configurations and access restrictions influence harm and evacuation counts. Results indicate that limited exit availability significantly increases harm, with classroom-adjacent exits playing a critical role in occupant survival. Additionally, findings reveal that certain architectural layouts, such as bar and radial designs, facilitate rapid shooter movement, leading to higher harm counts, whereas courtyard and cluster layouts limit shooter visibility and slow shooter progression, reducing harm rates. These insights highlight the significance of integrating spatial design considerations into school safety planning. The proposed RL-based simulation framework provides a scalable and generalizable approach for evaluating building safety, offering insights for architects, emergency planners, and policymakers to enhance school resilience against ASIs.
Keywords. Reinforcement learning, Active shooter incidents, School safety, Spatial resilience, Evacuation dynamics
Marti, D., Budathoki, A., Ding, Y., Lucas, G. M., & Nelson, D. (2025). How does acknowledging users’ preferences impact AI’s ability to make conflicting recommendations? International Journal of Human–Computer Interaction, 41(15), 9466–9477. https://doi.org/10.1080/10447318.2024.2426035 Show abstract
Artificial intelligence (AI) decision support systems are crucial in modern decision-making processes. Their increasing human-like adaptability introduces challenges, especially when their recommendations, for whatever reason, need to conflict with user preferences. This study examines the communication strategies AI systems should employ when their recommendations conflict with user preferences. We explored this research question through a hypothetical future interface where ChatGPT offers travel recommendations populated on a map. An online survey-based experiment was conducted, presenting 160 participants with ChatGPT-generated travel recommendations displayed alongside Bing map visuals. We employed a mixed-method experimental design, combining both between-subjects and within-subjects approaches, to investigate the impact of conflicting recommendations and the acknowledgment of user preferences on the acceptance of these recommendations. This effect is especially pronounced when the AI system acknowledges users’ preferences yet still offers conflicting recommendations to them. Contrary to the expectation that acknowledging users’ preferences could buffer the impact of such conflicts, our observations indicate the contrary. The presence of conflict following acknowledgment of users’ preferences, significantly causes a backfire effect, leading users to reject the recommendations. These findings underscore the need for consideration of recommendation delivery strategies in AI decision support systems and offer insights for designing future user interfaces and user experience research in the realm of recommendations provided by AI decision-support systems.
Keywords. Acknowledgement, AI-assisted decision-making, recommender systems, explanations, artificial intelligence, Large Language Models (LLM), user experience, HCI
Awada, M., Becerik-Gerber, B., Lucas, G. M., & Roll, S. C. (2025). The impact of color correlated temperature and illuminance levels of office lighting on stress and cognitive restoration. Journal of Environmental Psychology, 104, 102628. https://doi.org/10.1016/j.jenvp.2025.102628 Show abstract
In modern workplaces, the well-being and productivity of employees are increasingly recognized as essential for organizational success. This study explores the impact of lighting interventions—specifically, Correlated Color Temperature (CCT) and illumination intensity—on stress and cognitive function restoration among office workers. A between-subjects experimental design involving 100 participants assessed the effects of various lighting conditions on stress responses and cognitive performance after exposure to stress-inducing and cognitive fatigue tasks. Five experimental conditions were studied: 100 lux & 7000 K, 100 lux & 3000 K, 1000 lux & 7000 K, 1000 lux & 3000 K, and a baseline condition of 500 lux at 3700 K. Results indicated that lighting conditions significantly influence both physiological stress markers (such as skin conductance level and heart rate) and cognitive performance. Specifically, warm, and dimmed lighting (3000 K, 100 lux) effectively reduced stress markers and perceived stress levels, aligning with theories suggesting the calming effects of warmer white light tones. Conversely, cooler lighting (7000 K) was associated with enhanced cognitive performance and reduced cognitive fatigue, potentially due to its similarity to natural daylight, stimulating an alert state conducive to mental tasks. These findings suggest that lighting interventions offer a non-intrusive strategy to improve well-being and productivity in office environments, particularly addressing acute symptoms.
Seyedrezaei, M., Reyes, M., Becerik-Gerber, B., Roll, S. C., Lucas, G. M., & Narayanan, S. (2025, April). A longitudinal study of indoor environmental quality in hybrid workspaces [Poster presentation]. Southern California Civil and Environmental Engineering Research Symposium, Los Angeles, USA.
Brun, A., Lucas, G. M., & Becerik-Gerber, B. (2025). Under pressure: Contextualizing workplace stress towards user-centered interventions. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (pp. 1–9). New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/3706599.3719987 Show abstract
Stress is a pervasive challenge that significantly impacts worker health and well-being. Workplace stress is driven by various factors, ranging from organizational changes to poor workplace design. Although individual stress management strategies have been shown to be effective, current interventions often overlook personal and contextual factors shaping stress experiences. In this study, we conducted semi-structured interviews with eight office workers to gain a deeper understanding of their personal experiences with workplace stress. Our analysis reveals key stress triggers, coping mechanisms, and reflections on past stressful events. We highlight the multifaceted and individualized nature of workplace stress, emphasizing the importance of intervention timing, modality, and recognizing that stress is not solely a negative experience but can also have positive effects. Our findings provide actionable insights for the design of user-centered stress management solutions more attuned to the needs of office workers.
Keywords. Workplace stress, well-being, user-centered design, interventions, elicitation study
Liu, R., Becerik-Gerber, B., Pynadath, D. V., Marti, D., & Lucas, G. M. (2025). Elicitation and verification of learning via experts (EVOLVE) for creating a theoretical framework for active shooter incidents. Developments in the Built Environment, 21, 100635. https://doi.org/10.1016/j.dibe.2025.100635 Show abstract
Active Shooter Incidents (ASIs) pose significant threats to public safety, especially within educational facilities. Unlike fire safety regulations, current guidelines for ASIs lack specific recommendations. Thus, this study seeks to develop a theoretical framework for ASIs by examining factors influencing civilian behaviors, categorizing response behaviors, and identifying essential needs for building safety and emergency management. Employing the elicitation and verification of learning via experts (EVOLVE) method, we engaged experts through surveys, videos, prioritization exercises, and focus groups. We identified five categories of influencing factors: individual-related, building-related, situation-related, security-related, and shooter-related factors, and six civilian responses: running following a crowd, hiding in place, running then hiding, running independently, freezing, and fighting. Effective training and familiarity with building were found to be crucial for improving decisive actions and reducing passive behaviors. Experts emphasized the need for universal disaster response strategies, standardized training for first responders, and uniform civilian training in addition to understanding how various building design features enhance safety. Developing a risk assessment system and a simulation platform to enhance emergency management were also among the recommendations.
Keywords. Active shooter incidents, Campus safety, Civilian behavior, Influencing factors, Simulation, Elicitation, Building design
Liu, R., Becerik-Gerber, B., Lucas, G. M., & Busta, K. (2025). Impact of behavior-based virtual training on active shooter incident preparedness in healthcare facilities. International Journal of Disaster Risk Reduction, 118, 105225. https://doi.org/10.1016/j.ijdrr.2025.105225 Show abstract
In response to the increasing frequency of active shooter incidents (ASIs) in the United States and the vulnerability of healthcare facilities, this study examines the effectiveness of desktop-based virtual training in improving healthcare workers' preparedness for such events. A total of 97 participants were recruited to either use the training platform or engage with a control scenario. The training platform features a realistic virtual environment, a behavior-driven storyline, interactive elements, and an autonomous feedback system, all developed through stakeholder engagement. The control scenario was designed to match the training group's interaction exposure but replaced emergency-related content with exploration tasks while retaining the same interactive features. Both objective and subjective measures were used to assess healthcare workers' response performance, interpersonal support behaviors, and perceived preparedness. Additionally, the study explored the influence of personal factors—such as gender, age, occupation, sense of direction, previous training experience, and personal traits, including risk propensity—on the effectiveness of the training and participants' behaviors. The results indicate that the training platform significantly improves participants' response performance, interpersonal support behaviors, and perceived preparedness. Furthermore, the findings reveal differences in training effectiveness across gender, occupation, and prior training experience. Personal factors, including gender, personality, age, sense of direction, and previous training, play significant roles in shaping healthcare workers' preparedness for ASIs.
Keywords. Virtual reality, Training, Active shooter incidents, Personal factors, Simulation, Healthcare
Rodrigues, P. B., Wang, Z., Fang, Y., Becerik-Gerber, B., Soibelman, L., Lucas, G. M., & Roll, S. C. (2025). A teleoperation interface for robotic demolition: Designing for operators' sensory and adaptation needs [Paper presentation]. International Council for Research and Innovation in Building and Construction (CIB) World Building Congress, West Lafayette, IN.
Fang, Y., Aldrich, R., Lucas, G. M., & Roll, S. C. (2025). Exploring occupational adaptation in response to the evolving nature of work in the digital age [Paper presentation]. 23rd Annual Conference of the Society for the Study of Occupation: USA, Galveston, TX.
Wang, Z., Rodrigues, P. B., Fang, Y., Soibelman, L., Becerik-Gerber, B., Roll, S. C., & Lucas, G. M. (2025). Understanding potential challenges in demolition robot teleoperation to inform interface design: Insights from industry professionals. CIB Conferences, 1, 106. https://doi.org/10.7771/3067-4883.2040 Show abstract
Teleoperation is receiving intense attention due to its potential to address health and safety concerns for construction workers caused by on-site hazards. It allows operators to control robots from a distance outside their field of view using wireless communication technologies. Currently one of the most frequently deployed robotic technologies in construction is the remotely operated demolition robot within the field of view. While distance teleoperation offers great potential for enhancing worker safety and addressing labor shortages, the complex and dynamic demolition sites present unique challenges. This paper introduces the preliminary results of a study that explored potential challenges in demolition robot teleoperation, informed by the challenges identified in traditional demolition machine operations and the key concerns raised by industry professionals. A user-centered approach was employed through narrative interviews and focus groups with demolition professionals. The findings indicate that the challenges faced in traditional demolition machine operations may persist in teleoperation and could be further exacerbated by the sensory degradation inherent in teleoperation. Besides, the interactions among challenges increase the complexity of their overall impact. Moreover, enhancing operators’ situational awareness without inducing cognitive overload or distraction is critical for effectively addressing these challenges. Additionally, the results suggest the need for context-aware, multimodal teleoperation interfaces to assist operators in managing operational challenges. This study contributes to understanding challenges that operators may face during teleoperation, offering valuable insights for developing teleoperation interfaces adaptable to diverse demolition contexts.
Keywords. Challenges, Demolition, Teleoperation
Rodrigues, P. B., Becerik-Gerber, B., Soibelman, L., Lucas, G. M., & Roll, S. C. (2025). Impact of selective environmental sound attenuation on operator performance, stress, attention, and task engagement in teleoperated demolition. Automation in Construction, 169, 105876. https://doi.org/10.1016/j.autcon.2024.105876 Show abstract
The noise produced in demolition sites can mask safety-critical sounds that inform operators about task conditions and hazards. These problems are exacerbated in teleoperated demolition, where the separation between operator and site compromises operators' situation awareness and cognitive loads. This paper assessed the effects of environmental sounds with and without attenuation on the operators' performance and response (e.g., stress, attention, task engagement) during teleoperated demolition. Eighty participants completed three virtual demolition tasks under different environmental sound conditions, i.e., no sound (NS), unfiltered sound (US), and filtered sound (FS) with 20-dB attenuation of background noise and robot's sounds to allow focus on safety and task conditions. The results show that US induced more stress than NS and FS. Also, FS resulted in fewer collisions, faster reaction times, and greater attention and task engagement than US. These results can support the design of sound feedback interfaces for teleoperation in construction.
Keywords. Demolition, Electrodermal activity, Environmental sound, Heart rate variability, Performance, Robot, Safety, Stress, Teleoperation
Kwon, D., Weiss, E., Kulshrestha, T., Chawla, K., Lucas, G. M., & Gratch, J. (2024). Are LLMs effective negotiators? Systematic evaluation of the multifaceted capabilities of LLMs in negotiation dialogues. In Findings of the Association for Computational Linguistics: EMNLP 2024 (pp. 5391–5413). Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.findings-emnlp.310 Show abstract
A successful negotiation requires a range of capabilities, including comprehension of the conversation context, Theory-of-Mind (ToM) skills to infer the partner’s motives, strategic reasoning, and effective communication, making it challenging for automated systems. Despite the remarkable performance of LLMs in various NLP tasks, there is no systematic evaluation of their capabilities in negotiation. Such an evaluation is critical for advancing AI negotiation agents and negotiation research, ranging from designing dialogue systems to providing pedagogical feedback and scaling up data collection practices. This work aims to systematically analyze the multifaceted capabilities of LLMs across diverse dialogue scenarios throughout the stages of a typical negotiation interaction. Our analysis highlights GPT-4’s superior performance in many tasks while identifying specific challenges, such as making subjective assessments and generating contextually appropriate, strategically advantageous responses.
Kwon, D., Hae, J., & Lucas, G. M. (2024, November). Strategic moves: Negotiation agents with dynamic adaptation to opponent behavior for optimizing counter-offers [Poster presentation]. SoCal NLP Symposium, San Diego, USA.
Lucas, G. M., Becerik-Gerber, B., & Roll, S. C. (2024). Calibrating workers’ trust in intelligent automated systems. Patterns, 5(9), 101045. https://doi.org/10.1016/j.patter.2024.101045 Show abstract
With the exponential rise in the prevalence of automation, trust in such technology has become more critical than ever before. Trust is confidence in a particular entity, especially in regard to the consequences they can have for the trustor, and calibrated trust is the extent to which the judgments of trust are accurate. The focus of this paper is to reevaluate the general understanding of calibrating trust in automation, update this understanding, and apply it to worker’s trust in automation in the workplace. Seminal models of trust in automation were designed for automation that was already common in workforces, where the machine’s “intelligence” (i.e., capacity for decision making, cognition, and/or understanding) was limited. Now, burgeoning automation with more human-like intelligence is intended to be more interactive with workers, serving in roles such as decision aid, assistant, or collaborative coworker. Thus, we revise “calibrated trust in automation” to include more intelligent automated systems.
Keywords. trust, automation, calibrated trust, workers
Parga, M., Roll, S. C., Becerik-Gerber, B., Lucas, G. M., & Narayanan, S. (2024, September). Associations among social contexts of hybrid work and stress states: Implications for supporting worker performance [Poster presentation]. ASPIRE, Phoenix, USA.
Parga, M. R., Roll, S. C., Lucas, G. M., Becerik-Gerber, B., & Naranayan, S. (2024). Differences in self-rated worker outcomes across stress states: An interim analysis of hybrid worker data. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 68(1), 1404–1409. https://doi.org/10.1177/10711813241275500 Show abstract
Stress experiences can have dire consequences for worker performance and well-being, and the social environment of the workplace is a key contributor to worker experience. This study investigated the relationship between hybrid workers’ self-ratings of productivity, mood, and stress with perceptions of positive (eustress) and negative (distress) stress states. We hypothesized that self-ratings would vary across combinations of eustress and distress experiences and that these differences would differ based on the social context. Ecological momentary assessments (EMA) were used to obtain ecologically valid data at four data points each workday across a 4-month study period in a cohort of seven office workers. Findings aligned with the Yerkes–Dodson law, such that higher states of arousal were associated with greater self-perceived productivity, and higher stress magnitudes were found when distress existed. Compared to other states, eustress was associated with higher productivity in work-related activities and better mood across all activity types.
Fischer, K., Velentza, A.-M., Lucas, G. M., & Williams, D. (2024). Seeing eye to eye with robots: An experimental study predicting trust in social robots for domestic use. In 2024 33rd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN) (pp. 2162–2168). IEEE. https://doi.org/10.1109/RO-MAN60168.2024.10731371 Show abstract
The use of social robots in service tasks is spreading, showcasing advantages for both consumers and service providers. However, their widespread adoption is hindered by a notable lack of trust. Our study aims to uncover insights into the factors influencing the adoption of social robots in home settings, exploring the factors that lead users to trust and eventually adopt robots. We designed two experimental conditions, presenting the Amazon Astro robot from different perspectives (high-angle and eye-level) and demonstrating its different abilities to 198 people recruited from MTurk. We employed both quantitative (trust, first impressions of warmth and competence as well as usability, familiarity, and attitudes) questionnaires and qualitative (word analysis) assessments, and results showed that participants had higher trust scores when seeing the robot from an eye-level perspective. In addition, usability, familiarity and competence were shown to explain a significant amount of variance in trust. While existing negative attitudes towards robots and the participants’ age were shown to be the strongest predictors for participants’ willingness to purchase a robot, trust was able to significantly affect use intention. We contribute to the broader understanding of the challenges and opportunities in integrating social robots into daily life, shedding light on the dynamics between technological innovation and consumer adoption.
Keywords. Social robots, trust, willingness to buy, use intention, Amazon Astro
Liu, R., Wu, W., Becerik-Gerber, B., & Lucas, G. M. (2024). Enhancing building safety design for active shooter incidents: Exploration of building exit parameters using reinforcement learning-based simulations. In Proceedings of the 31st International Workshop on Intelligent Computing in Engineering (EG-ICE 2024) (pp. 569–579). European Group for Intelligent Computing in Engineering. Full text
Awada, M., Becerik-Gerber, B., Lucas, G., Roll, S., & Liu, R. (2024). A new perspective on stress detection: An automated approach for detecting eustress and distress. IEEE Transactions on Affective Computing, 15(3), 1153–1165. https://doi.org/10.1109/TAFFC.2023.3324910 Show abstract
Previous studies have solely focused on establishing Machine Learning (ML) models for automated detection of stress arousal. However, these studies do not recognize stress appraisal and presume stress is a negative mental state. Yet, stress can be classified according to its influence on individuals; the way people perceive a stressor determines whether the stress reaction is considered as eustress (positive stress) or distress (negative stress). Thus, this study aims to assess the potential of using an ML approach to determine stress appraisal and identify eustress and distress instances using physiological and behavioral features. The results indicate that distress leads to higher perceived stress arousal compared to eustress. An XGBoost model that combined physiological and behavioral features using a 30 second time window had 83.38% and 78.79% F1-scores for predicting eustress and distress, respectively. Gender-based models resulted in an average increase of 2-4% in eustress and distress prediction accuracy. Finally, a model to predict the simultaneous assessment of eustress and distress, distinguishing between pure eustress, pure distress, eustress-distress coexistence, and the absence of stress achieved a moderate F1-score of 65.12%. The results of this study lay the foundation for work management interventions to maximize eustress and minimize distress in the workplace.
Brun, A., Lucas, G. M., Becerik-Gerber, B., Roll, S. C., & Narayanan, S. (2024). Toward occupant-centric virtual agents: Office workers’ perspectives on technological design and implementation. In Proceedings of the 31st International Workshop on Intelligent Computing in Engineering (EG-ICE 2024) (pp. 104–114). European Group for Intelligent Computing in Engineering. Full text
Chawla, K., Clever, R., Ramirez, J., Lucas, G. M., & Gratch, J. (2024). Towards emotion-aware agents for improved user satisfaction and partner perception in negotiation dialogues. IEEE Transactions on Affective Computing, 15(2), 433–444. https://doi.org/10.1109/TAFFC.2023.3238007 Show abstract
Negotiation is a complex social interaction that encapsulates emotional encounters in human decision-making. Virtual agents that can negotiate with humans by the means of language are useful in pedagogy and conversational AI. To advance the development of such agents, we explore the role of emotion in the prediction of two important subjective goals in a negotiation – outcome satisfaction and partner perception. We devise ways to measure and compare different degrees of emotion expression in negotiation dialogues, consisting of emoticon, lexical, and contextual variables. Through an extensive analysis of a large-scale dataset in chat-based negotiations, we find that incorporating emotion expression explains significantly more variance, above and beyond the demographics and personality traits of the participants. Further, our temporal analysis reveals that emotive information from both early and later stages of the negotiation contributes to this prediction, indicating the need for a continual learning model of capturing emotion for automated agents. Finally, we extend our analysis to another dataset, showing promise that our findings generalize to more complex scenarios. We conclude by discussing our insights, which will be helpful for designing adaptive negotiation agents that interact through realistic communication interfaces.
Keywords. Emotion recognition, human-agent interaction, multi-issue bargaining, natural language processing, negotiation dialogues, partner perception, T5 transformer model, user satisfaction
Rodrigues, P. B., Becerik-Gerber, B., Soibelman, L., Lucas, G. M., & Roll, S. C. (2024). Virtual environment for studying the effects of operational and environmental sounds on teleoperated demolition. In Computing in Civil Engineering 2023 (pp. 54–61). Reston, VA: American Society of Civil Engineers. https://doi.org/10.1061/9780784485231.007 Show abstract
Teleoperated robots are increasingly being used in construction sites, but there is still a need for better human-robot interfaces. Auditory feedback using sonification has commonly been used to complement visual feedback in teleoperation applications. However, it is not yet well understood how operational and environmental sounds can support operators in complex tasks such as construction machine operations. In this paper, we describe the development of a virtual environment to assess the impacts of operational and environmental sounds on operators’ performances and cognitive loads during teleoperated demolition. Our goal is to examine whether different levels of operational and environmental sounds can increase the workers’ situation awareness, operational efficiency, and safety performance. Alternatively, we anticipate that some environmental noises may negatively affect the operators’ performances by increasing their mental workload and stress levels. Consequently, we plan to evaluate the effectiveness of including audio filters that minimize unwanted sounds while prioritizing desired sounds.
Liu, R., Becerik-Gerber, B., Lucas, G. M., & Busta, K. (2024). Development of a VR training platform for active shooter incident preparedness in healthcare environments via a stakeholder-engaged process. In Computing in Civil Engineering 2023 (pp. 45–53). Reston, VA: American Society of Civil Engineers. https://doi.org/10.1061/9780784485231.006 Show abstract
Healthcare workers face a unique set of challenges during active shooter incidents. Due to job responsibilities and mobility issues, healthcare workers and patients are vulnerable in these types of scenarios. This study aims to develop a VR training platform to prepare healthcare workers for active shooter incidents. To achieve the goals of this study, we went through a collaborative process with stakeholders to identify the unique circumstances for healthcare workers and environments. To achieve the desired characteristics for the platform, we followed a four-component design framework consisting of the development of a representative virtual environment, a behavior-based storyline, interactive features, and an autonomous feedback system. In describing the detailed design of the four components, this paper presents a case study for researchers who are seeking to develop platforms that need user adaption and acceptance.
Awada, M., Becerik Gerber, B., Lucas, G. M., & Roll, S. C. (2024). Stress appraisal in the workplace and its associations with productivity and mood: Insights from a multimodal machine learning analysis. PLoS ONE, 19(1), e0296468. https://doi.org/10.1371/journal.pone.0296468 Show abstract
Previous studies have primarily focused on predicting stress arousal, encompassing physiological, behavioral, and psychological responses to stressors, while neglecting the examination of stress appraisal. Stress appraisal involves the cognitive evaluation of a situation as stressful or non-stressful, and as a threat/pressure or a challenge/opportunity. In this study, we investigated several research questions related to the association between states of stress appraisal (i.e., boredom, eustress, coexisting eustress-distress, distress) and various factors such as stress levels, mood, productivity, physiological and behavioral responses, as well as the most effective ML algorithms and data signals for predicting stress appraisal. The results support the Yerkes-Dodson law, showing that a moderate stress level is associated with increased productivity and positive mood, while low and high levels of stress are related to decreased productivity and negative mood, with distress overpowering eustress when they coexist. Changes in stress appraisal relative to physiological and behavioral features were examined through the lenses of stress arousal, activity engagement, and performance. An XGBOOST model achieved the best prediction accuracies of stress appraisal, reaching 82.78% when combining physiological and behavioral features and 79.55% using only the physiological dataset. The small accuracy difference of 3% indicates that physiological data alone may be adequate to accurately predict stress appraisal, and the feature importance results identified electrodermal activity, skin temperature, and blood volume pulse as the most useful physiologic features. Implementing these models within work environments can serve as a foundation for designing workplace policies, practices, and stress management strategies that prioritize the promotion of eustress while reducing distress and boredom. Such efforts can foster a supportive work environment to enhance employee well-being and productivity.
Fang, Y., Roll, S. C., Becerik-Gerber, B., Lucas, G. M., & Soibelman, L. (2024). Using a community-engaged approach to develop a safe and effective teleoperation workstation in construction [Paper presentation]. ASPIRE - International Meeting of the Human Factors and Ergonomics Society, Phoenix, AZ.
Chawla, K., Wu, I., Rong, Y., Lucas, G. M., & Gratch, J. (2023). Be selfish, but wisely: Investigating the impact of agent personality in mixed-motive human-agent interactions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 13078–13092). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.emnlp-main.808 Show abstract
A natural way to design a negotiation dialogue system is via self-play RL: train an agent that learns to maximize its performance by interacting with a simulated user that has been designed to imitate human-human dialogue data. Although this procedure has been adopted in prior work, we find that it results in a fundamentally flawed system that fails to learn the value of compromise in a negotiation, which can often lead to no agreements (i.e., the partner walking away without a deal), ultimately hurting the model’s overall performance. We investigate this observation in the context of DealOrNoDeal task, a multi-issue negotiation over books, hats, and balls. Grounded in negotiation theory from Economics, we modify the training procedure in two novel ways to design agents with diverse personalities and analyze their performance with human partners. We find that although both techniques show promise, a selfish agent, which maximizes its own performance while also avoiding walkaways, performs superior to other variants by implicitly learning to generate value for both itself and the negotiation partner. We discuss the implications of our findings for what it means to be a successful negotiation dialogue system and how these systems should be designed in the future.
Awada, M., Becerik-Gerber, B., Lucas, G., & Roll, S. (2023). Predicting office workers’ productivity: A machine learning approach integrating physiological, behavioral, and psychological indicators. Sensors, 23(21), 8694. https://doi.org/10.3390/s23218694 Show abstract
This research pioneers the application of a machine learning framework to predict the perceived productivity of office workers using physiological, behavioral, and psychological features. Two approaches were compared: the baseline model, predicting productivity based on physiological and behavioral characteristics, and the extended model, incorporating predictions of psychological states such as stress, eustress, distress, and mood. Various machine learning models were utilized and compared to assess their predictive accuracy for psychological states and productivity, with XGBoost emerging as the top performer. The extended model outperformed the baseline model, achieving an R2 of 0.60 and a lower MAE of 10.52, compared to the baseline model’s R2 of 0.48 and MAE of 16.62. The extended model’s feature importance analysis revealed valuable insights into the key predictors of productivity, shedding light on the role of psychological states in the prediction process. Notably, mood and eustress emerged as significant predictors of productivity. Physiological and behavioral features, including skin temperature, electrodermal activity, facial movements, and wrist acceleration, were also identified. Lastly, a comparative analysis revealed that wearable devices (Empatica E4 and H10 Polar) outperformed workstation addons (Kinect camera and computer-usage monitoring application) in predicting productivity, emphasizing the potential utility of wearable devices as an independent tool for assessment of productivity. Implementing the model within smart workstations allows for adaptable environments that boost productivity and overall well-being among office workers.
Keywords. productivity; stress; mood; eustress; distress; psychological state; physiological features; behavioral features
Liu, R., Awada, M., Becerik-Gerber, B., Lucas, G. M., & Roll, S. C. (2023). Gender moderates the effects of ambient bergamot scent on stress restoration in offices. Journal of Environmental Psychology, 91, 102135. https://doi.org/10.1016/j.jenvp.2023.102135 Show abstract
We investigated the physiological (heart rate variability) and psychological (state of anxiety, pleasantness, and comfort) effects of ambient bergamot scent on the stress levels of office workers by exposing them to the scent while stressors persisted as the workers continued to work on the office tasks. Forty-eight young adults were randomly assigned to either a control or scent group. Our results show that the stress restoration effect of bergamot scent depends on gender. The change in heart rate variability revealed that bergamot scent increased stress among males but not for females. The reported pleasantness and comfort followed the same trend. Compared to the control groups, females in the scent group thought the office smelled pleasant and felt more comfortable, but males in the scent group reported the opposite. However, no gender effect was found in the level of state anxiety. Specifically, compared to the control groups, both males and females exposed to the bergamot scent self-reported decreasing stress levels. This inconsistency between self-reported stress and physiological measurements is not uncommon, especially among males who are socialized to downplay emotional experiences. Our data suggest that there is indeed a gender difference in the effectiveness of the bergamot scent for reducing stress in office workers.
Tak, A. N., Becerik-Gerber, B., Soibelman, L., & Lucas, G. M. (2023). A framework for investigating the acceptance of smart home technologies: Findings for residential smart HVAC systems. Building and Environment, 245, 110935. https://doi.org/10.1016/j.buildenv.2023.110935 Show abstract
We developed a framework to investigate the drivers for and barriers against adopting integrated smart home technologies to encourage broader adoption of these technologies that can significantly reduce the environmental impact of residential buildings. The framework entails an extension of the second version of the United Theory of Acceptance and Use of Technology (UTAUT2) as the theoretical foundation for modeling acceptance behavior. The framework was validated by conducting an online survey of 1024 potential users focusing on smart home HVAC systems, utilizing structural equation modeling for analysis. The results demonstrated that three UTAUT2 constructs (i.e., performance expectancy, effort expectancy, and habit), as well as trust from the model extension, exert significant effects on the adoption intent. We also found that privacy and transparency are predictors of trust. Adequacy and customizability of interaction exhibited a strong effect on effort expectancy. The proposed framework can be adopted in future efforts to study the acceptance of other smart home technologies in a broader and more diverse demographic.
Keywords. Smart home, Technology acceptance, HVAC systems, Smart thermostats, UTAUT
Wu, I., Rong, Y., Chalwa, K., Lucas, G. M., & Gratch, J. (2023, November). Negotiation agents with interpretable strategic planning: Synergy of LLMs and reinforcement learning-based steering [Poster presentation]. SoCal NLP Symposium, Los Angeles, USA.
Chawla, K., Kwon, D., Weiss, E., Kulshrestha, T., Lucas, G. M., & Gratch, J. (2023, November). Are LLMs effective negotiators? Evaluating the multifaceted capabilities of LLMs in negotiation dialogues [Poster presentation]. SoCal NLP Symposium, Los Angeles, USA.
Seyedrezaei, M., Awada, M., Becerik-Gerber, B., Lucas, G., & Roll, S. (2023). Interaction effects of indoor environmental quality factors on cognitive performance and perceived comfort of young adults in open plan offices in North American Mediterranean climate. Building and Environment, 244, 110743. https://doi.org/10.1016/j.buildenv.2023.110743 Show abstract
While Indoor Environmental Quality (IEQ) factors in an environment co-exist, the interaction effects of these factors and their impacts on cognitive functioning and perceived comfort have not been comprehensively examined. In this study, the interaction effects between temperature, lighting Correlated Color Temperature (CCT), and noise levels on selective attention, sustained attention, creativity, acoustics, thermal, visual, and overall IEQ comfort of young adults in open-plan offices in North American Mediterranean climate were presented. In a mixed-design controlled experimental setting, 52 young adults were recruited, and their objective cognitive performance and subjective comfort were assessed through statistical analysis. The experimental set points included [20 °C, 25 °C], [2700 K, 6500 K], and [50 dB, 65 dB] for temperature, lighting color, and noise, respectively. Additionally, the work took into consideration the gender and Body Mass Index (BMI) of participants. The results show that temperature moderated the effect of noise level and lighting CCT on selective attention, while no effect of IEQ factors on sustained attention was found. Creativity was influenced by gender and its interaction with the noise level. Concerning perceived comfort, acoustic comfort varied significantly with temperature. Thermal comfort was influenced by the combined moderating effect of lighting CCT and BMI on temperature, while visual comfort was driven by the moderation effect of gender on lighting CCT. Overall comfort was affected by the noise level and temperature. Finally, cognitive performance indicators were correlated with perceived IEQ comfort votes. Based on the findings of this study, considerations of interactions between noise, lighting CCT, temperature, gender, and BMI can shape occupant-centric priorities for enhanced cognitive functioning and comfort.
Liu, R., Zhu, R., Becerik-Gerber, B., Lucas, G. M., & Southers, E. G. (2023). Be prepared: How training and emergency type affect evacuation behaviour. Journal of Computer Assisted Learning, 39(5), 1493–1509. https://doi.org/10.1111/jcal.12812 Show abstract
Background. Video-based training has been widely adopted by private organizations and public authorities to educate occupants on various types of building emergencies. However, the effectiveness of video-based training for preparing occupants for building emergencies has not been rigorously studied nor has the impact of emergency type been investigated on training effectiveness.
Objectives. This study examines whether video-based training is an effective method to prepare occupants for building emergencies and how the effectiveness differs in the context of different building emergencies.
Methods. We simulated fire and active shooter emergencies in a virtual office building and conducted evacuation experiments to examine participants' emergency responses using both objective and subjective metrics. A total of 108 participants were recruited and responded to the fire or active shooter incident with or without video-based training.
Results and Conclusions. The results revealed that participants with video-based training more often chose to follow other recommendations when responding to building emergencies instead of simply following others. Results from ANOVA showed that training increased participants' self-efficacy significantly, especially for those in the active shooter group. Moreover, participants in the active shooter simulation had a higher level of response efficacy than those in the fire emergency simulation. Our results also demonstrated the influence of emergency type on participants' final decisions and considerations of the recommendations.
Implications. Our results suggested that video-based training is effective in improving participants' emergency preparedness and changing their behaviour patterns to a certain extent such as reducing following behaviour and encouraging safe evacuations. Additionally, statistically significant interactions between video-based training and emergency types suggested that training effectiveness should be considered in accordance with the emergency type.
Junghaenel, D. U., Schneider, S., Lucas, G. M., Boberg, J., Weinstein, F. M., Richeimer, S. H., Stone, A. A., & Lumley, M. A. (2023). Virtual human–delivered interviews for patients with chronic pain: Feasibility, acceptability, and a pilot randomized trial of standard medical, psychosocial, and educational interviews. Psychosomatic Medicine, 85(7), 627–638. https://doi.org/10.1097/PSY.0000000000001228 Show abstract
Objective. Seminal advances in virtual human (VH) technology have introduced highly interactive, computer-animated VH interviewers. Their utility for aiding in chronic pain care is unknown. We developed three interactive telehealth VH interviews-a standard pain-focused, a psychosocial risk factor, and a pain psychology and neuroscience educational interview. We then conducted a preliminary investigation of their feasibility, acceptability, and efficacy. We also experimentally compared a human and a computer-generated VH voice.
Methods. Patients (N = 94, age = 22-78 years) with chronic musculoskeletal pain were randomly assigned to the standard (n = 31), psychosocial (n = 34), or educational (n = 29) VH interview and one of the two VH voices. Acceptability ratings included patient satisfaction and expectations/evaluations of the VH interview. Outcomes assessed at baseline and about 1-month postinterview were pain intensity, interference, emotional distress, pain catastrophizing, and readiness for pain self-management. Linear mixed-effects models were used to test between- and within-condition effects.
Results. Acceptability ratings showed that satisfaction with the VH and telehealth format was generally high, with no condition differences. Study attrition was low (n = 5). Intent-to-treat-analyses showed that, compared with the standard interview, the psychosocial interview yielded a significantly greater reduction in pain interference (p = .049, d = 0.43) and a marginally greater reduction in pain intensity (p = .054, d = 0.36), whereas the educational interview led to a marginally greater yet nonsignificant increase in readiness for change (p = .095, d = 0.24), as well as several significant improvements within-condition. Results did not differ by VH voice.
Conclusions. Interactive VH interviewers hold promise for improving chronic pain care, including probing for psychosocial risk factors and providing pain-related education.
Hoegen, J., Lucas, G. M., Shore, D., Parkinson, B., & Gratch, J. (2023). How expression and context determine second-person judgments of emotion. In 2023 11th International Conference on Affective Computing and Intelligent Interaction (ACII) (pp. 1–7). IEEE. https://doi.org/10.1109/ACII59096.2023.10388189 Show abstract
Within the field of Affective Computing, facial expressions have traditionally been used as a means of inferring valence and emotions. Most studies have focused on interpreting facial expressions as an isolated signal, typically training algorithms with annotators with a 3rd-person point of view, often without access to the original context. When the context is provided, recent research highlights that the interpretation annotators assign to facial expressions is sometimes more strongly influenced by the context than the facial expression. But even when the context is provided, annotators are psychologically and physically distant from the original setting that produced the emotion. In this paper, we explore how context and facial expressions shape 2nd-person interpretation of facial expressions and compare this to 1st-person self-report. Results show that both expression and context contribute to self and other impressions but in different ways. Expressions and context are independent predictors of 1st-person judgments but interact to determine 2nd. person judgments. In particular, the way players interpret their partner’s facial cues changes dramatically based on what just occurred in the game. We discuss the implication of these findings for automatic emotion recognition methods.
Keywords. context, facial expression, valence, prisoner’s dilemma
Liu, R., Becerik-Gerber, B., & Lucas, G. M. (2023). Effectiveness of VR-based training on improving occupants’ response and preparedness for active shooter incidents. Safety Science, 164, 106175. https://doi.org/10.1016/j.ssci.2023.106175 Show abstract
Effective training for preparing building occupants for active shooter incidents is essential to reduce harm and fatalities. Virtual Reality (VR) technology has been increasingly used as a promising training method in many applications including emergency preparedness. However, previous studies for active shooter incidents have mainly focused on developing VR-based training prototypes. Thus, the effectiveness of VR-based training for active shooter incidents has yet to be rigorously studied. This study focuses on the effectiveness of VR-based training for active shooter preparedness compared to traditional training methods and investigates the factors behind the improved performance. To study the effectiveness of VR-based training, a total of 108 participants were recruited to experience video-based training, VR-based training with low interactivity, and VR-based training with high interactivity. Both objective and subjective measures were assessed. The results show that VR-based training is more effective than video-based training in improving participants’ response performance and perceived preparedness for active shooter incidents. The immersion and interactivity of the VR-based training contribute to performance. Participants without previous training experience benefit more from VR-based training than participants who previously had received some training to prepare for active shooter incidents.
Keywords. Virtual reality, Video, Training, Active shooter incidents, Occupants, Simulation
Rodrigues, P. B., Singh, R., Oytun, M., Adami, P., Woods, P. J., Becerik-Gerber, B., Soibelman, L., Copur-Gencturk, Y., & Lucas, G. M. (2023). A multidimensional taxonomy for human-robot interaction in construction. Automation in Construction, 150, 104845. https://doi.org/10.1016/j.autcon.2023.104845 Show abstract
Despite the increased interest in construction robotics both in academia and the industry, insufficient attention has been given to aspects related to Human-Robot Interaction (HRI). Characterizing HRI for construction tasks can help researchers organize knowledge in a structured manner that allows for classifying construction robotics applications and comparing and benchmarking different studies. This paper builds upon existing taxonomies and empirical studies in HRI in various industries (e.g., construction, manufacturing, and military, among others) to propose a multidimensional taxonomy to characterize HRI applications in the construction industry. The taxonomy design followed a systematic literature review in which common themes were identified and grouped into 16 categories. The proposed taxonomy can be used as a foundation for systematic reviews and meta-analyses of HRI applications in construction and can benefit the construction industry by informing the design of collaborative tasks performed by human-robot teams.
Keywords. Collaborative tasks, Construction, Human-Robot Interaction (HRI), Taxonomy
Chawla, K., Shi, W., Zhang, J., Lucas, G. M., Yu, Z., & Gratch, J. (2023). Social influence dialogue systems: A survey of datasets and models for social influence tasks. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics (pp. 750–766). Dubrovnik, Croatia: Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.eacl-main.53 Show abstract
Dialogue systems capable of social influence such as persuasion, negotiation, and therapy, are essential for extending the use of technology to numerous realistic scenarios. However, existing research primarily focuses on either task-oriented or open-domain scenarios, a categorization that has been inadequate for capturing influence skills systematically. There exists no formal definition or category for dialogue systems with these skills and data-driven efforts in this direction are highly limited. In this work, we formally define and introduce the category of social influence dialogue systems that influence users’ cognitive and emotional responses, leading to changes in thoughts, opinions, and behaviors through natural conversations. We present a survey of various tasks, datasets, and methods, compiling the progress across seven diverse domains. We discuss the commonalities and differences between the examined systems, identify limitations, and recommend future directions. This study serves as a comprehensive reference for social influence dialogue systems to inspire more dedicated research and discussion in this emerging area.
Awada, M., Becerik-Gerber, B., Liu, R., Seyedrezaei, M., Lu, Z., Xenakis, M., Lucas, G., Roll, S. C., & Narayanan, S. (2023). Ten questions concerning the impact of environmental stress on office workers. Building and Environment, 229, 109964. https://doi.org/10.1016/j.buildenv.2022.109964 Show abstract
We regularly face stress during our everyday activities, to the extent that stress is recognized by the World Health Organization as the epidemic of the 21st century. Stress is how humans respond physically and psychologically to adjustments, experiences, conditions, and circumstances in their lives. While there are many reasons for stress, work and job pressure remain the main cause. Thus, companies are increasingly interested in creating healthier, more comfortable, and stress-free offices for their workers. The indoor environment can induce environmental stress when it cannot satisfy the individual needs for health and comfort. In fact, office environmental conditions (e.g., thermal, and indoor air conditions, lighting, and noise) and interior design parameters (e.g., office layout, colors, furniture, access to views, distance to window, personal control and biophilic design) have been found to affect office workers' stress levels. A line of research based on the stress recovery theory offers new insights for establishing offices that limit environmental stress and help with work stress recovery. To that end, this paper answers ten questions that explore the relation between the indoor office-built environment and stress levels among workers. The answers to the ten questions are based on an extensive literature review to draw conclusions from what has been achieved to date. Thus, this study presents a foundation for future environmental stress related research in offices.
Keywords. Stress; Indoor environmental quality; Stress recovery; Office; Interior design
Lucas, G. M., Mell, J., Boberg, J., Zenone, F., de Visser, E. J., Tossell, C., & Seech, T. (2023). Customizing virtual interpersonal skills training applications may not improve trainee performance. Scientific Reports, 13, 78. https://doi.org/10.1038/s41598-022-27154-2 Show abstract
While some theoretical perspectives imply that the context of a virtual training should be customized to match the intended context where those skills would ultimately be applied, others suggest this might not be necessary for learning. It is important to determine whether manipulating context matters for performance in training applications because customized virtual training systems made for specific use cases are more costly than generic "off-the-shelf" ones designed for a broader set of users. Accordingly, we report a study where military cadets use a virtual platform to practice their negotiation skills, and are randomly assigned to one of two virtual context conditions: military versus civilian. Out of 28 measures capturing performance in the negotiation, there was only one significant result: cadets in the civilian condition politely ask the agent to make an offer significantly more than those in the military condition. These results imply that—for this interpersonal skills application, and perhaps ones like it—virtual context may matter very little for performance during social skills training, and that commercial systems may yield real benefits to military scenarios with little-to-no modification.
Adami, P., Singh, R., Borges Rodrigues, P., Becerik-Gerber, B., Soibelman, L., Copur-Gencturk, Y., & Lucas, G. M. (2023). Participants matter: Effectiveness of VR-based training on the knowledge, trust in the robot, and self-efficacy of construction workers and university students. Advanced Engineering Informatics, 55, 101837. https://doi.org/10.1016/j.aei.2022.101837 Show abstract
Virtual Reality (VR)-based training has gained attention from the scientific community in the Architecture, Engineering, and Construction (AEC) industry as a cost-effective and safe method that eliminates the safety risks that may impose on workers during the training compared to traditional training methods (e.g., in-person hands-on training, apprenticeship). Although researchers have developed VR-based training for construction workers, some have recruited students rather than workers to understand the effect of their VR-based training. However, students are different from construction workers in many ways, which can threaten the validity of such studies. Hence, research is needed to investigate the extent to which the findings of a VR-based training study are contingent on whether students or construction workers were used as the study sample. This paper strives to compare the effectiveness of VR-based training on university students’ and construction workers’ knowledge acquisition, trust in the robot, and robot operation self-efficacy in remote operation of a construction robot. Twenty-five construction workers and twenty-five graduate construction engineering students were recruited to complete a VR-based training for remote operating a demolition robot. We used quantitative analyses to answer our research questions. Our study shows that the results are dependent on the target sample in that students gained more knowledge, whereas construction workers gained more trust in the robot and more self-efficacy in robot operation. These findings suggest that the effectiveness of VR-based training on students may not necessarily associate with its effectiveness on construction workers.
Keywords. Construction robotics, VR-based training, Human-robot interaction, Trust in the robot, Robot operation self-efficacy, Knowledge acquisition
Roll, S. C., Becerik-Gerber, B., & Lucas, G. M. (2023). Developing a framework for the safe and effective development and implementation of artificial intelligence in the workplace [Webinar Presentation]. Artificial Intelligence Interest Group of the National Institute for Occupational Safety and Health; virtual event.
Holbrook, C., Holman, D., Wagner, A. R., Marghetis, T., Lucas, G. M., Sheeran, B., Surendran, V., Armagost, J., Spazak, S., Andor, K., & Yin, Y. (2023). Investigating human-robot overtrust during crises. In Proceedings of the Workshops at the Second International Conference on Hybrid Human-Artificial Intelligence (HHAI 2023) (3456, pp. 164–168). CEUR-WS.org. Full text Show abstract
Our research focuses on human-robot interaction (HRI) during life-or-death emergencies. We have developed an immersive virtual reality (VR) testbed because conducting real-world crisis simulations would pose prohibitive logistical difficulties, as well as to leverage the affordances of VR technology to measure motor behavior (e.g., distance maintained between self and robot), information foraging (e.g., as indexed by headset movement variability and eyetracking), or autonomic arousal (e.g., as indexed by shifts in pupil dilation or grip strength). Findings to date using minimally haptic VR confirm that participants treat the simulated active-shooter crisis seriously, and act in ways which validly mirror prior studies of real-world HRI under threat. We will describe these methods, including our manipulation of robot anthropomorphism and our current move to integrate full-body haptics to maximize both experiential immersion and incentives to avoid pain.
Keywords. human-robot interaction, decision-making, virtual reality, trust
Becerik-Gerber, B., Lucas, G., Aryal, A., Awada, M., Bergés, M., Billington, S., Boric-Lubecke, O., Ghahramani, A., Heydarian, A., Höelscher, C., Jazizadeh, F., Khan, A., Langevin, J., Liu, R., Marks, F., Mauriello, M. L., Murnane, E., Noh, H., Pritoni, M., Roll, S., Schaumann, D., Seyedrezaei, M., Taylor, J. E., Zhao, J., & Zhu, R. (2022). The field of human building interaction for convergent research and innovation for intelligent built environments. Scientific Reports, 12, 22092. https://doi.org/10.1038/s41598-022-25047-y Show abstract
Human-Building Interaction (HBI) is a convergent field that represents the growing complexities of the dynamic interplay between human experience and intelligence within built environments. This paper provides core definitions, research dimensions, and an overall vision for the future of HBI as developed through consensus among 25 interdisciplinary experts in a series of facilitated workshops. Three primary areas contribute to and require attention in HBI research: humans (human experiences, performance, and well-being), buildings (building design and operations), and technologies (sensing, inference, and awareness). Three critical interdisciplinary research domains intersect these areas: control systems and decision making, trust and collaboration, and modeling and simulation. Finally, at the core, it is vital for HBI research to center on and support equity, privacy, and sustainability. Compelling research questions are posed for each primary area, research domain, and core principle. State-of-the-art methods used in HBI studies are discussed, and examples of original research are offered to illustrate opportunities for the advancement of HBI research.
Keywords. Civil engineering; Environmental impact; Mechanical engineering; Occupational health; Quality of life
Becerik-Gerber, B., Lucas, G., Aryal, A., Awada, M., Bergés, M., Billington, S. L., Boric-Lubecke, O., Ghahramani, A., Heydarian, A., Jazizadeh, F., Liu, R., Zhu, R., Marks, F., Roll, S., Seyedrezaei, M., Taylor, J. E., Höelscher, C., Khan, A., Langevin, J., Mauriello, M. L., Murnane, E., Noh, H., Pritoni, M., Schaumann, D., & Zhao, J. (2022). Ten questions concerning human-building interaction research for improving the quality of life. Building and Environment, 226, 109681. https://doi.org/10.1016/j.buildenv.2022.109681 Show abstract
This paper seeks to address ten questions that explore the burgeoning field of Human-Building Interaction (HBI), an interdisciplinary field that represents the next frontier in convergent research and innovation to enable the dynamic interplay of human and building interactional intelligence. The field of HBI builds on several existing efforts in historically separate research fields/communities and aims to understand how buildings affect human outcomes and experiences, as well as how humans interact with, adapt to, and affect the built environment and its systems, to support buildings that can learn, enable adaptation, and evolve at different scales to improve the quality-of-life of its users while optimizing resource usage and service availability. Questions were developed by a diverse group of researchers with backgrounds in design, engineering, computer science, social science, and health science. Answers to these questions draw conclusions from what has been achieved to date as reported in the available literature and establish a foundation for future HBI research. This paper aims to encourage interdisciplinary collaborations in HBI research to change the way people interact with and perceive technology within the context of buildings and inform the design, construction, and operation of next-generation, intelligent built environments. In doing so, HBI research can realize a myriad of benefits for human users, including improved productivity, health, cognition, convenience, and comfort, all of which are essential to societal well-being.
Keywords. Building lifecycle; Human-centered; Occupants; Built environment; Well-being; Interaction; Quality of life
Pauw, L. S., Sauter, D. A., van Kleef, G. A., Lucas, G. M., Gratch, J., & Fischer, A. H. (2022). The avatar will see you now: Support from a virtual human provides socio-emotional benefits. Computers in Human Behavior, 136, 107368. https://doi.org/10.1016/j.chb.2022.107368 Show abstract
When people are in emotional distress, they often seek support. Virtual humans may provide unique and complementary benefits to human support provision, given that virtual humans are readily available and help sharers overcome socio-evaluative threats, thereby increasing willingness to disclose. Here, we examined whether talking to a virtual human elicits socio-emotional benefits, and whether this is moderated by the type of support provided. To examine the scope of the potential effect, we compared two key types of support (emotional and cognitive), across two emotions (anger and worry). Participants (N = 115) shared two personal emotional experiences with a virtual human, who provided either emotional or cognitive support via the Wizard-of-Oz method (i.e., a human-operated avatar). Results showed that participants felt better after talking to the virtual human, as evidenced by reduced intensity of the target emotion and generally improved affect. The emotional improvement was similar for emotional and cognitive support. Cognitive support was also experienced as equally effective as emotional support, and led to similar levels of experienced closeness and desire to interact with the virtual human again. These findings suggest that talking to a virtual human can be a valuable form of support at times of distress.
Keywords. Virtual human, Interpersonal emotion regulation, Social support, Emotional support, Cognitive support, Computer-human interaction
Awada, M., Becerik-Gerber, B., Lucas, G., & Roll, S. (2022). Cognitive performance, creativity and stress levels of neurotypical young adults under different white noise levels. Scientific Reports, 12, 14566. https://doi.org/10.1038/s41598-022-18862-w Show abstract
Noise is often considered a distractor; however recent studies suggest that sub-attentive individuals or individuals diagnosed with attention deficit hyperactivity disorder can benefit from white noise to enhance their cognitive performance. Research regarding the effect of white noise on neurotypical adults presents mixed results, thus the implications of white noise on the neurotypical population remain unclear. Thus, this study investigates the effect of 2 white noise conditions, white noise level at 45 dB and white noise level at 65 dB, on the cognitive performance, creativity, and stress levels of neurotypical young adults in a private office space. These conditions are compared to a baseline condition where participants are exposed to the office ambient noise. Our findings showed that the white noise level at 45 dB resulted in better cognitive performance in terms of sustained attention, accuracy, and speed of performance as well as enhanced creativity and lower stress levels. On the other hand, the 65 dB white noise condition led to improved working memory but higher stress levels, which leads to the conclusion that different tasks might require different noise levels for optimal performance. These results lay the foundation for the integration of white noise into office workspaces as a tool to enhance office workers’ performance.
Fukumura, Y. E., Roll, S. C., Xiao, Y., Awada, M., Schott, J., Becerik-Gerber, B., & Lucas, G. M. (2022, August). Healthy occupational engagement in built environments: Lessons learned from work-from-home transitions during COVID-19 [Paper presentation]. 18th World Federation of Occupational Therapists International Congress and Exhibition, Paris, France.
Chawla, K., Lucas, G. M., May, J., & Gratch, J. (2022). Opponent modeling in negotiation dialogues by related data adaptation. In Findings of the Association for Computational Linguistics: NAACL 2022 (pp. 661–674). Association for Computational Linguistics. https://doi.org/10.18653/v1/2022.findings-naacl.50 Show abstract
Opponent modeling is the task of inferring another party’s mental state within the context of social interactions. In a multi-issue negotiation, it involves inferring the relative importance that the opponent assigns to each issue under discussion, which is crucial for finding high-value deals. A practical model for this task needs to infer these priorities of the opponent on the fly based on partial dialogues as input, without needing additional annotations for training. In this work, we propose a ranker for identifying these priorities from negotiation dialogues. The model takes in a partial dialogue as input and predicts the priority order of the opponent. We further devise ways to adapt related data sources for this task to provide more explicit supervision for incorporating the opponent’s preferences and offers, as a proxy to relying on granular utterance-level annotations. We show the utility of our proposed approach through extensive experiments based on two dialogue datasets. We find that the proposed data adaptations lead to strong performance in zero-shot and few-shot scenarios. Moreover, they allow the model to perform better than baselines while accessing fewer utterances from the opponent. We release our code to support future work in this direction.
Adami, P., Rodrigues, P. B., Woods, P. J., Becerik-Gerber, B., Soibelman, L., Copur-Gencturk, Y., & Lucas, G. M. (2022). Impact of VR-based training on human–robot interaction for remote operating construction robots. Journal of Computing in Civil Engineering, 36(3), 04022006. https://doi.org/10.1061/(ASCE)CP.1943-5487.0001016 Show abstract
Despite the increased interest in automation and the expanded deployment of robots in the construction industry, using robots in a dynamic and unstructured working environment has caused safety concerns in operating construction robots. Improving human–robot interaction (HRI) can increase the adoption of robots on construction sites; for example, increasing trust in robots could help construction workers to accept new technologies. Confidence in operation (or self-efficacy), mental workload, and situational awareness are among other key factors that help such workers to remote operate robots safely. However, construction workers have very few opportunities to practice with robots to build trust, self-efficacy, and situational awareness, as well as resistance against increasing mental workload, before interacting with them on job sites. Virtual reality (VR) could afford a safer place to practice with the robot; thus, we tested if VR-based training could improve these four outcomes during the remote operation of construction robots. We measured trust in the robot, self-efficacy, mental workload, and situational awareness in an experimental study where construction workers remote-operated a demolition robot. Fifty workers were randomly assigned to either VR-based training or traditional in-person training led by an expert trainer. Results show that VR-based training significantly increased trust in the robot, self-efficacy, and situational awareness, compared to traditional in-person training. Our findings suggest that VR-based training can allow for significant increases in beneficial cognitive factors over more traditional methods and has substantial implications for improving HRI using VR, especially in the construction industry.
Rodrigues, P. B., Xiao, Y., Fukumura, Y. E., Awada, M., Aryal, A., Becerik-Gerber, B., Lucas, G., & Roll, S. C. (2022). Ergonomic assessment of office worker postures using 3D automated joint angle assessment. Advanced Engineering Informatics, 52, 101596. https://doi.org/10.1016/j.aei.2022.101596 Show abstract
Sedentary activity and static postures are associated with work-related musculoskeletal disorders (WMSDs) and worker discomfort. Ergonomic evaluation for office workers is commonly performed by experts using tools such as the Rapid Upper Limb Assessment (RULA), but there is limited evidence suggesting sustained compliance with expert’s recommendations. Assessing postural shifts across a day and identifying poor postures would benefit from automation by means of real-time, continuous feedback. Automated postural assessment methods exist; however, they are usually based on ideal conditions that may restrict users’ postures, clothing, and hair styles, or may require unobstructed views of the participants. Using a Microsoft Kinect camera and open-source computer vision algorithms, we propose an automated ergonomic assessment algorithm to monitor office worker postures, the 3D Automated Joint Angle Assessment, 3D-AJA. The validity of the 3D-AJA was tested by comparing algorithm-calculated joint angles to the angles obtained from manual goniometry and the Kinect Software Development Kit (SDK) for 20 participants in an office space. The results of the assessment show that the 3D-AJA has mean absolute errors ranging from 5.6° ± 5.1° to 8.5° ± 8.1° for shoulder flexion, shoulder abduction, and elbow flexion relative to joint angle measurements from goniometry. Additionally, the 3D-AJA showed relatively good performance on the classification of RULA score A using a Random Forest model (micro averages F1-score = 0.759, G-mean = 0.811), even at high levels of occlusion on the subjects’ lower limbs. The results of the study provide a basis for the development of a full-body ergonomic assessment for office workers, which can support personalized behavior change and help office workers to adjust their postures, thus reducing their risks of WMSDs.
Keywords. Ergonomic assessment; RULA; Engineering office environments; Depth camera; Computer vision; Machine learning
Zhu, R., Lucas, G. M., Becerik-Gerber, B., Southers, E. G., & Landicho, E. (2022). The impact of security countermeasures on human behavior during active shooter incidents. Scientific Reports, 12, 929. https://doi.org/10.1038/s41598-022-04922-8 Show abstract
Active shooter incidents represent an increasing threat to American society, especially in commercial and educational buildings. In recent years, a wide variety of security countermeasures have been recommended by public and governmental agencies. Many of these countermeasures are aimed to increase building security, yet their impact on human behavior when an active shooter incident occurs remains underexplored. To fill this research gap, we conducted virtual experiments to evaluate the impact of countermeasures on human behavior during active shooter incidents. A total of 162 office workers and middle/high school teachers were recruited to respond to an active shooter incident in virtual office and school buildings with or without the implementation of multiple countermeasures. The experiment results showed countermeasures significantly influenced participants' response time and decisions (e.g., run, hide, fight). Participants' responses and perceptions of the active shooter incident were also contingent on their daily roles, as well as building and social contexts. Teachers had more concerns for occupants' safety than office workers. Moreover, teachers had more positive perceptions of occupants in the school, whereas office workers had more positive perceptions of occupants in the office.
Fukumura, Y. E., Kijel, M. T., Xiao, Y., Becerik-Gerber, B., Lucas, G. M., & Roll, S. C. (2022). Artificial intelligence to support worker health, well-being, and participation: Using an occupational lens for workplace technology development [Paper presentation]. 18th World Federation of Occupational Therapists International Congress and Exhibition, Paris, France.
Becerik-Gerber, B., Lucas, G. M., & Roll, S. C. (2022). AI and other technologies to support health and wellbeing in the workplace [Platform presentation]. Marconi Conference of the Office Ergonomics Research Committee, Atlanta, GA.
Roll, S. C., Becerik-Gerber, B., & Lucas, G. M. (2022). Leveraging a transdisciplinary approach to develop effective AI solutions to reduce risk and promote worker health [Webinar presentation]. Artificial Intelligence Interest Group of the National Institute for Occupational Safety and Health; virtual event.
Awada, M., Becerik-Gerber, B., Lucas, G., & Roll, S. (2021). Associations among home indoor environmental quality factors and worker health while working from home during COVID-19 pandemic. Journal of Engineering for Sustainable Buildings and Cities, 2(4), 041001. https://doi.org/10.1115/1.4052822 Show abstract
The outbreak of SARS-CoV-2 virus forced office workers to conduct their daily work activities from home over an extended period. Given this unique situation, an opportunity emerged to study the satisfaction of office workers with indoor environmental quality (IEQ) factors of their houses where work activities took place and associate these factors with mental and physical health. We designed and administered a questionnaire that was open for 45 days during the COVID-19 pandemic and received valid data from 988 respondents. The results show that low satisfaction with natural lighting, glare and humidity predicted eye related symptoms, while low satisfaction with noise was a strong predictor of fatigue or tiredness, headaches or migraines, anxiety, and depression or sadness. Nose and throat related symptoms and skin related symptoms were only uniquely predicted by low satisfaction with humidity. Low satisfaction with glare uniquely predicted an increase in musculoskeletal discomfort. Symptoms related to mental stress, rumination or worry were predicted by low satisfaction with air quality and noise. Finally, low satisfaction with noise and indoor temperature predicted the prevalence of symptoms related to trouble concentrating, maintaining attention or focus. Workers with higher income were more satisfied with humidity, air quality and indoor temperature and had better overall mental health. Older individuals had increased satisfaction with natural lighting, humidity, air quality, noise, and indoor temperature. Findings from this study can inform future design practices that focus on hybrid home-work environments by highlighting the impact of IEQ factors on occupant well-being.
Fukumura, Y. E., Schott, J. M., Lucas, G. M., Becerik-Gerber, B., & Roll, S. C. (2021). Negotiating time and space when working from home: Experiences during COVID-19. OTJR: Occupation, Participation and Health, 41(4), 223-231. https://doi.org/10.1177/15394492211033830 Show abstract
Stay-at-home mandates following the COVID-19 pandemic increased work from home (WFH). While WFH offers many benefits, navigating work in nontraditional contexts can be a challenge. The objective of this study was to explore the benefits and challenges of WFH during COVID-19 to identify supports and resources necessary. Comments from two free-response questions on a survey regarding experiences of WFH (N = 648, N = 366) were analyzed using inductive qualitative content analysis. Four themes emerged: time use, considerations of working in the home space, intersections between work-life and home-life, and temporality of WFH as situated within a pandemic. Across all themes were concerns related to participation in both work and home roles, work performance, and well-being. Findings highlight the importance of support during times of disruption of occupational patterns, roles, and routines. Despite challenges, many individuals hoped to continue WFH. Organizations should consider the complex intersections of work-life and home-life to develop supportive policies and resources.
Keywords. work; survey; context; health; occupational balance
Toyoda, Y., Lucas, G. M., & Gratch, J. (2021). Predicting worker accuracy from nonverbal behaviour: Benefits and potential for algorithmic bias. In Companion Publication of the 2021 International Conference on Multimodal Interaction (pp. 25–30). Association for Computing Machinery. https://doi.org/10.1145/3461615.3485427 Show abstract
With the rise of algorithmic management (and online work in general), there is a growing interest in techniques that can monitor worker performance. For example, if a system can automatically detect whether a worker is becoming distracted or disengaged, it can intervene to motivate the worker or flag their output as requiring further quality control. Prior research has explored the potential for detecting nonverbal cues that could predict mistakes (e.g., detecting boredom in students or fatigue in drivers). Here, we learn a model that reliably predicts worker accuracy from nonverbal behaviour during tedious and repetitive annotation tasks. We show that the annotation accuracy can be substantially improved by discarding annotations that are predicted to be of low accuracy by the model. While these results are promising, recent concerns about algorithmic bias led us to further investigate whether the accuracy is influenced by skin tone. Unfortunately, we find that the algorithm showed systematic bias that disadvantaged some dark-skinned workers and incorrectly rewarded some with lighter-skin. We discuss the apparent reasons for this bias and suggestions for how and if such methods could be deployed to enhance worker engagement.
Keywords. worker accuracy, facial action unit, gaze information, algorithmic management, algorithmic bias
Adami, P., Rodrigues, P. B., Woods, P. J., Becerik-Gerber, B., Soibelman, L., Copur-Gencturk, Y., & Lucas, G. M. (2021). Effectiveness of VR-based training on improving construction workers’ knowledge, skills, and safety behavior in robotic teleoperation. Advanced Engineering Informatics, 50, 101431. https://doi.org/10.1016/j.aei.2021.101431 Show abstract
The emergence of construction robotics and automation has produced an urgent and vast need for construction workers to reskill and upskill for the future of work. Virtual Reality (VR)-based training has been considered and investigated as a safe and cost-effective training method that allows workers to be exposed to hazardous tasks with negligible actual safety risks in comparison to existing training methods (hands-on, lecture-based, apprenticeship training). This paper aims to investigate the impact of VR-based training on construction workers’ knowledge acquisition, operational skills, and safety behavior during robotic teleoperation compared to the traditional in-person training method. Fifty construction workers were randomly assigned to complete either VR-based or in-person training for operating a demolition robot. We used quantitative and qualitative data analyses to answer our research questions. Our results indicate that VR-based training was associated with a significant increase in knowledge, operational skills, and safety behavior compared to in-person training. Our findings suggest that VR-based training not only provides a viable and effective option for future training programs but a valuable option for construction robotics safety and skill training.
Keywords. Construction robotics, VR-based training, Human-robot interaction, Safety behavior, Operational skills, Knowledge acquisition
Mell, J., Lucas, G. M., & Gratch, J. (2021). Pandemic panic: The effect of disaster-related stress on negotiation outcomes. In Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents (pp. 148–155). Association for Computing Machinery. https://doi.org/10.1145/3472306.3478353 Show abstract
Prior research often finds increased altruism following natural disasters. One explanation is the social heuristic hypothesis: humans are prosocial by nature but become self-interested when they have the opportunity to deliberate. As the stress of a disaster lowers people's ability to engage in effortful deliberation, their heuristic prosocial tendencies emerge. However, this link has often been explored with very simple tasks like the dictator game. Here we study the impact of COVID-related stress on outcomes in multi-issue negotiations with a computational virtual agent. These tasks are interesting because they share some of the characteristics of dictator games (some pot of resources must be divided) but they also involve presumably effortful perspective taking (that can grow the size of the pot). Furthermore, the interaction of humans with virtual agents allows us to explore the extent to which humans apply the CASA (computers as social actors) paradigm to negotiation when under considerable stress. In two experiments with a virtual negotiation partner, we provide evidence for two distinct pathways for how COVID-19 stress shapes prosocial behavior. Consistent with the social heuristic hypothesis, COVID-stress increases giving, mediated by heuristic thinking. But COVID-stress also seems to enhance information-exchange and perspective taking, which allowed participants to grow more value which they could give away. Our results give new insights into the relationship between stress, cognition, and prosocial behavior.
Keywords. Negotiation, Human-Computer Interaction, IAGO Platform, Stress, COVID-19, Dual-Process Cognition, Empirical Study, Mediation Analysis
Johnson, E., Gratch, J., Boberg, J., DeVault, D., Kim, P., & Lucas, G. M. (2021). Using intelligent agents to examine gender in negotiations. In Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents (pp. 90–97). Association for Computing Machinery. https://doi.org/10.1145/3472306.3478348 Show abstract
Women earn less than men in technical fields. Competing theories have been offered to explain this disparity. Some argue that women underperform in negotiating their salary, in-part due to language in job descriptions, called gender triggers, which leave women feeling disadvantaged in salary negotiations. Others point to structural and institutional bias: i.e., recruiters make better offers to men even when women exhibit equal negotiation skills. As a final salary is co-constructed though an interaction between employees and recruiters, it is difficult to disentangle these views. Here, we discuss how intelligent virtual agents serve as powerful methodological tools that lend new insight into this psychological debate. We use virtual negotiators to examine the impact of gender triggers on computer science (CS) undergraduates that engaged in a simulated salary negotiation with an automated recruiter. We find that, regardless of gender, CS students are reluctant to negotiate, and this hesitancy likely lowers their starting salary. Even when they negotiate, students show little skill in discovering tradeoffs that could enhance their salary, highlighting the need for negotiation training in technical fields. Most importantly, we find little evidence that gender triggers impact women's negotiated outcomes, at least within the field of CS. We argue that findings that emphasize women's individual deficits may reflect a lack of experimental control, which intelligent agents can help correct, and that structural and institutional explanations of inequity deserve greater attention.
Keywords. Negotiation, gender, bias, methodological tools
Chawla, K., Clever, R., Ramirez, J., Lucas, G. M., & Gratch, J. (2021). Towards emotion-aware agents for negotiation dialogues. In 2021 9th International Conference on Affective Computing and Intelligent Interaction (ACII) (pp. 1–8). IEEE. https://doi.org/10.1109/ACII52823.2021.9597427 Show abstract
Negotiation is a complex social interaction that encapsulates emotional encounters in human decision-making. Virtual agents that can negotiate with humans are useful in pedagogy and conversational AI. To advance the development of such agents, we explore the prediction of two important subjective goals in a negotiation – outcome satisfaction and partner perception. Specifically, we analyze the extent to which emotion attributes extracted from the negotiation help in the prediction, above and beyond the individual difference variables. We focus on a recent dataset in chat-based negotiations, grounded in a realistic camping scenario. We study three degrees of emotion dimensions – emoticons, lexical, and contextual by leveraging affective lexicons and a state-of-the-art deep learning architecture. Our insights will be helpful in designing adaptive negotiation agents that interact through realistic communication interfaces.
Keywords. negotiation dialogues, human-agent interaction, emotion models, outcome prediction
Awada, M., Lucas, G., Becerik-Gerber, B., & Roll, S. (2021). Working from home during the COVID-19 pandemic: Impact on office worker productivity and work experience. Work, 69(4), 1171-1189. https://doi.org/10.3233/WOR-210301 Show abstract
Background. With the COVID-19 pandemic, organizations embraced Work From Home (WFH). An important component of transitioning to WFH is the effect on workers, particularly related to their productivity and work experience.
Objectives. The objective of this study is to examine how worker-, workspace-, and work-related factors affected productivity and time spent at a workstation on a typical WFH day during the pandemic.
Methods. An online questionnaire was designed and administered to collect the necessary information. Data from 988 respondents were included in the analyses.
Results. Overall perception of productivity level among workers did not change relative to their in-office productivity before the pandemic. Female, older, and high-income workers were likely to report increased productivity. Productivity was positively influenced by better mental and physical health statuses, having a teenager, increased communication with coworkers and having a dedicated room for work. Number of hours spent at a workstation increased by approximately 1.5 hours during a typical WFH day. Longer hours were reported by individuals who had school age children, owned an office desk or an adjustable chair, and had adjusted their work hours.
Conclusion. The findings highlight key factors for employers and employees to consider for improving the WFH experience.
Fukumura, Y. E., McLaughlin Gray, J., Lucas, G., Becerik-Gerber, B., & Roll, S. C. (2021). Office worker perspective on an artificial intelligence workstation: A qualitative study. American Journal of Occupational Therapy, 75(Supplement_2), 7512505154. https://doi.org/10.5014/ajot.2021.75S2-RP154 Show abstract
Accepted for AOTA INSPIRE 2021 but unable to be presented due to online event limitations.
This study explored office workers' perspectives on including artificial intelligence (AI) in their office workspace. Following an iterative analysis of six focus-group interviews with a total of 45 participants, three constructs emerged. Rich discussions demonstrated how acceptability of an AI workstation is complex and affected by the person, context, and their occupations.
Chawla, K., Ramirez, J., Clever, R., Lucas, G. M., May, J., & Gratch, J. (2021). CaSiNo: A corpus of campsite negotiation dialogues for automatic negotiation systems. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 3167–3185). Association for Computational Linguistics. https://doi.org/10.18653/v1/2021.naacl-main.254 Show abstract
Automated systems that negotiate with humans have broad applications in pedagogy and conversational AI. To advance the development of practical negotiation systems, we present CaSiNo: a novel corpus of over a thousand negotiation dialogues in English. Participants take the role of campsite neighbors and negotiate for food, water, and firewood packages for their upcoming trip. Our design results in diverse and linguistically rich negotiations while maintaining a tractable, closed-domain environment. Inspired by the literature in human-human negotiations, we annotate persuasion strategies and perform correlation analysis to understand how the dialogue behaviors are associated with the negotiation performance. We further propose and evaluate a multi-task framework to recognize these strategies in a given utterance. We find that multi-task learning substantially improves the performance for all strategy labels, especially for the ones that are the most skewed. We release the dataset, annotations, and the code to propel future work in human-machine negotiations: https://github.com/kushalchawla/CaSiNo
Lee, M., Lucas, G. M., & Gratch, J. (2021). Comparing mind perception in strategic exchanges: Human-agent negotiation, dictator and ultimatum games. Journal on Multimodal User Interfaces, 15(2), 201–214. https://doi.org/10.1007/s12193-020-00356-6 Show abstract
Recent research shows that how we respond to other social actors depends on what sort of mind we ascribe to them. In a comparative manner, we observed how perceived minds of agents shape people’s behavior in the dictator game, ultimatum game, and negotiation against artificial agents. To do so, we varied agents’ minds on two dimensions of the mind perception theory: agency (cognitive aptitude) and patiency (affective aptitude) via descriptions and dialogs. In our first study, agents with emotional capacity garnered more allocations in the dictator game, but in the ultimatum game, agents’ described agency and affective capacity, both led to greater offers. In the second study on negotiation, agents ascribed with low-agency traits earned more points than those with high-agency traits, though the negotiation tactic was the same for all agents. Although patiency did not impact game points, participants sent more happy and surprise emojis and emotionally valenced messages to agents that demonstrated emotional capacity during negotiations. Further, our exploratory analyses indicate that people related only to agents with perceived affective aptitude across all games. Both perceived agency and affective capacity contributed to moral standing after dictator and ultimatum games. But after negotiations, only agents with perceived affective capacity were granted moral standing. Manipulating mind dimensions of machines has differing effects on how people react to them in dictator and ultimatum games, compared to a more complex economic exchange like negotiation. We discuss these results, which show that agents are perceived not only as social actors, but as intentional actors through negotiations, in contrast with simple economic games.
Keywords. Mind perception theory, Theory of mind, human-agent negotiation, Dictator game, Ultimatum game, Virtual agent, Robot
Aryal, A., Becerik-Gerber, B., Lucas, G. M., & Roll, S. C. (2021). Intelligent agents to improve thermal satisfaction by controlling personal comfort systems under different levels of automation. IEEE Internet of Things Journal, 8(8), 7089-7100. https://doi.org/10.1109/JIOT.2020.3038378 Show abstract
Heating, ventilation, and air conditioning (HVAC) systems account for 43% of building energy consumption, yet only 38% of commercial building occupants are satisfied with the thermal environment. The primary reasons for low occupant satisfaction are that HVAC operations do not integrate occupant comfort requirements nor control the thermal environment at the individual level. Personal comfort systems (PCS) enable local control of the thermal environment around each occupant. However, full manual control of PCS can be inefficient, and fully automated PCS reduces an occupant’s perceived control over the environment, which can then lead to lower satisfaction. A better solution might lie somewhere between fully manual and fully automated environmental control. In this paper, we describe the development and implementation of an internet of things (IoT) based intelligent agent that learns individual occupant comfort requirements and controls the thermal environment using PCS (i.e., a local fan and a heater). We tested different levels of automation where control is shared between an intelligent agent and the end user. Our results show that PCS use improves occupant satisfaction and including some level of automation can improve occupant satisfaction further than what is possible with manually operated PCS. Among the levels of automation investigated, Inquisitive Automation, where the user approves/declines the control actions of the intelligent agent before execution, led to highest occupant satisfaction with the thermal environment.
Keywords. building automation, thermal comfort, smart systems, smart buildings, indoor environments
Xiao, Y., Becerik-Gerber, B., Lucas, G., & Roll, S. C. (2021). Impacts of working from home during COVID-19 pandemic on physical and mental well-being of office workstation users. Journal of Occupational and Environmental Medicine, 63(3), 181-190. https://doi.org/10.1097/JOM.0000000000002097 Show abstract
Objective. To understand impacts of social, behavioral and physical factors on well-being of office workstation users during COVID-19 work from home (WFH).
Methods. A questionnaire was deployed from April 24 to June 11, 2020 and 988 responses were valid. Linear regression, multinomial logistic regression and chi-square tests were used to understand factors associated with overall physical and mental health statuses and number of new physical and mental health issues.
Results. Decreased overall physical and mental well-being after WFH were associated with physical exercise, food intake, communication with coworkers, children at home, distractions while working, adjusted work hours, workstation set-up and satisfaction with workspace indoor environmental factors.
Conclusion. This study highlights factors that impact workers’ physical and mental health well-being while WFH and provides a foundation for considering how to best support a positive WFH experience.
Keywords. COVID-19 pandemic, home office, mental well-being, physical well-being, work from home
Fukumura, Y. E., McLaughlin Gray, J., Lucas, G. M., Becerik-Gerber, B., & Roll, S. C. (2021). Worker perspectives on incorporating artificial intelligence into office workspaces: Implications for the future of office work. International Journal of Environmental Research and Public Health, 1(4), 1690. https://doi.org/10.3390/ijerph18041690 Show abstract
Workplace environments have a significant impact on worker performance, health, and well-being. With machine learning capabilities, artificial intelligence (AI) can be developed to automate individualized adjustments to work environments (e.g., lighting, temperature) and to facilitate healthier worker behaviors (e.g., posture). Worker perspectives on incorporating AI into office workspaces are largely unexplored. Thus, the purpose of this study was to explore office workers’ views on including AI in their office workspace. Six focus group interviews with a total of 45 participants were conducted. Interview questions were designed to generate discussion on benefits, challenges, and pragmatic considerations for incorporating AI into office settings. Sessions were audio-recorded, transcribed, and analyzed using an iterative approach. Two primary constructs emerged. First, participants shared perspectives related to preferences and concerns regarding communication and interactions with the technology. Second, numerous conversations highlighted the dualistic nature of a system that collects large amounts of data; that is, the potential benefits for behavior change to improve health and the pitfalls of trust and privacy. Across both constructs, there was an overarching discussion related to the intersections of AI with the complexity of work performance. Numerous thoughts were shared relative to future AI solutions that could enhance the office workplace. This study’s findings indicate that the acceptability of AI in the workplace is complex and dependent upon the benefits outweighing the potential detriments. Office worker needs are complex and diverse, and AI systems should aim to accommodate individual needs.
Keywords. workspace; office work; computer workstations; artificial intelligence
Awada, M., Zhu, R., Becerik-Gerber, B., Lucas, G. M., & Southers, E. (2021). An integrated emotional and physiological assessment for VR-based active shooter incident experiments. Advanced Engineering Informatics, 47, 101227. https://doi.org/10.1016/j.aei.2020.101227 Show abstract
Unfortunately, active shooter incidents are on the rise in the United States. With the recent technological advancements, virtual reality (VR) experiments could serve as an effective method to prepare civilians and law enforcement personnel for such scenarios. However, for VR experiments to be effective for active shooter training and research, such experiments must be able to evoke emotional and physiological responses as live active shooter drills and events do. The objective of this study is thus to test the effectiveness of an active shooter VR experiment on emotional and physiological responses. Additionally, we consider different locomotion techniques (i.e., walk-in-place and controller) and explore their impact on users’ sense of presence. The results suggest that the VR active shooter experiment in this study can induce emotional arousal and increase heart rate of the participants immersed in the virtual environment. Furthermore, compared to the controller, the walk-in-place technique resulted in a higher emotional arousal in terms of negative emotions and a stronger sense of presence. The study presents a foundation for future active shooter experiments as it supports the ecological validity using VR for active shooter incident related work for the purposes of training or research.
Keywords. Active shooter incidents, Virtual reality, Emotional response, Physiological response, Locomotion technique
Gratch, J., & Lucas, G. M. (2021). Emotionally resonant media: Advances in sensing, understanding, and influencing human emotion through interactive media. In K. Döveling & E. A. Konijn (Eds.), Routledge international handbook of emotions and media (2nd ed., pp. 285–302). London, UK: Routledge. https://doi.org/10.4324/9780429465758-18 Show abstract
Science fiction has long predicted that technology will develop relationships and interact with humans using emotions, just like other humans do. Technologies that can use emotions in conversations with people are now becoming reality. To do so, technology needs to sense and respond to human emotions and intentions. First, technology should be capable of perception – to understand signals given by the human user, including words, quality of speech and accompanying nonverbal cues such as facial expressions, gestures and posture. The agent would then need to be able to reason about these cues and in relation to its own goals and the users. Finally, the technology needs to be able to translate its resulting decisions into suitable and emotionally rich behaviors. The chapter reviews work on each of these steps in turn: progress toward technology being able to perceive human emotional cues, reason about emotion and generate emotional cues. Finally, we discuss relevant applications for entertainment, education, health and communication.
Angelika-Nikita, M., de Melo, C. M., Terada, K., Lucas, G. M., & Gratch, J. (2021). The impact of partner expressions on felt emotion in the iterated prisoner’s dilemma: An event-level analysis. In Proceedings of the Ninth Annual Conference on Advances in Cognitive Systems (pp. 1–18). Cognitive Systems Foundation. https://doi.org/10.48550/arXiv.2207.00925 Show abstract
Social games like the prisoner's dilemma are often used to develop models of the role of emotion in social decision-making. Here we examine an understudied aspect of emotion in such games: how an individual's feelings are shaped by their partner's expressions. Prior research has tended to focus on other aspects of emotion. Research on felt-emotion has focused on how an individual's feelings shape how they treat their partner, or whether these feelings are authentically expressed. Research on expressed-emotion has focused on how an individual's decisions are shaped by their partner's expressions, without regard for whether these expressions actually evoke feelings. Here, we use computer-generated characters to examine how an individual's moment-to-moment feelings are shaped by (1) how they are treated by their partner and (2) what their partner expresses during this treatment. Surprisingly, we find that partner expressions are far more important than actions in determining self-reported feelings. In other words, our partner can behave in a selfish and exploitive way, but if they show a collaborative pattern of expressions, we will feel greater pleasure collaborating with them. These results also emphasize the importance of context in determining how someone will feel in response to an expression (i.e., knowing a partner is happy is insufficient; we must know what they are happy-at). We discuss the implications of this work for cognitive-system design, emotion theory, and methodological practice in affective computing.
Fukumura, Y. E., Becerik-Gerber, B., Lucas, G. M., & Roll, S. C. (2021). Impact of COVID-19 on work: The transactional nature of work from home [Paper presentation]. 27th USC Chan Occupational Science Symposium; virtual event.
Gratch, J., & Lucas, G. M. (2021). Rapport between humans and socially interactive agents. In B. Lugrin, C. Pelachaud, & D. Traum (Eds.), The handbook on socially interactive agents: 20 years of research on embodied conversational agents, intelligent virtual agents, and social robotics: Vol. 1. Methods, behavior, cognition (pp. 433–462). New York, NY: Association for Computing Machinery. https://doi.org/10.1145/3477322.3477335
Adami, P., Becerik-Gerber, B., Soibelman, L., Doleck, T., Copur-Gencturk, Y., & Lucas, G. M. (2020). An immersive virtual learning environment for worker-robot collaboration on construction sites. In 2020 Winter Simulation Conference (WSC) (pp. 2400–2411). IEEE. https://doi.org/10.1109/WSC48552.2020.9383944 Show abstract
This paper presents a Virtual Learning Environment (VLE) that simulates the collaboration between a construction robot and a construction worker in order to support training in worker-robot teamwork on construction sites. The VLE presented in the paper utilizes a semi-autonomous remote-controlled demolition robot named Brokk. This paper describes the use-case robot (Brokk), system setup and configuration, and the learning scenarios. In this VLE, the trainee learns about the robot’s various components, safety management, functions of the robot's control box, and general guidelines used during common demolition tasks. Importantly, this VLE is developed based on adult learning theories in general— and andragogy principles in particular. In addition, we augment the VLE with useful features from existing VLEs. The effectiveness of the developed VLE will be evaluated with construction workers based on knowledge and performance assessments.
Mell, J., Lucas, G. M., & Gratch, J. (2020). Varied magnitude favor exchange in human-agent negotiation. In Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents (pp. 1–8). Association for Computing Machinery. https://doi.org/10.1145/3383652.3423866 Show abstract
Agents that interact with humans in complex, social tasks need the ability to comprehend as well as employ common social strategies. In negotiation, there is ample evidence of such techniques being used efficaciously in human interchanges. In this work, we demonstrate a new design for socially aware agents that employ one such technique—favor exchange—in order to gain value when playing against humans. In an online study of a robust, simulated social negotiation task, we show that these agents are effective against real human participants. In particular, we show that agents that ask for favors during the course of a repeated set of negotiations are more successful than those that do not. Additionally, previous work has demonstrated that humans can detect when agents betray them by failing to return favors that were previously promised. By contrast, this work indicates that these betrayal techniques may go largely undetected in complex scenarios.
Zhu, R., Lucas, G. M., Becerik-Gerber, B., & Southers, E. G. (2020). Building preparedness in response to active shooter incidents: Results of focus group interviews. International Journal of Disaster Risk Reduction, 48, 101617. https://doi.org/10.1016/j.ijdrr.2020.101617 Show abstract
Active shooter incidents present an increasing threat to the American society. Many of these incidents occur in building environments, therefore, it is important to consider design and security elements in buildings to decrease the risk of active shooter incidents. This study aims to assess current security countermeasures and identify varying considerations associated with implementing these countermeasures. Fifteen participants, with expertise and experience in a diverse collection of operational and organizational backgrounds, including security, engineering, law enforcement, emergency management and policy making, participated in three focus group interviews. The participants identified a list of countermeasures that have been used for active shooter incidents. Important determinants for the effectiveness of countermeasures include their influence on occupants' behavior during active shooter incidents, and occupants' and administrators' awareness of how to use them effectively. The nature of incidents (e.g., internal vs. external threats), building type (e.g., office buildings vs. school buildings), and occupants (e.g., students of different ages) were also recognized to affect the selection of appropriate countermeasures. The nexus between emergency preparedness and normal operations, and the importance of tradeoffs such as the ones between cost, aesthetics, maintenance needs and the influence on occupants' daily activities were also discussed. To ensure the effectiveness of countermeasures and improve safety, the participants highlighted the importance of both training and practice, for occupants and administrators (e.g., first responder teams). The interview results suggested that further study of the relationship between security countermeasures and occupants' and administrators’ responses, as well as efficient training approaches are needed.
Keywords. Active shooter incidents, Focus group interviews, Building preparedness, Countermeasures, Evaluation
Mell, J., Lucas, G. M., Mozgai, S., & Gratch, J. (2020). The effects of experience on deception in human-agent negotiation. Journal of Artificial Intelligence Research, 68, 633–660. https://doi.org/10.1613/jair.1.11924 Show abstract
Negotiation is the complex social process by which multiple parties come to mutual agreement over a series of issues. As such, it has proven to be a key challenge problem for designing adequately social AIs that can effectively navigate this space. Artificial AI agents that are capable of negotiating must be capable of realizing policies and strategies that govern offer acceptances, offer generation, preference elicitation, and more. But the next generation of agents must also adapt to reflect their users’ experiences.
The best human negotiators tend to have honed their craft through hours of practice and experience. But, not all negotiators agree on which strategic tactics to use, and endorsement of deceptive tactics in particular is a controversial topic for many negotiators. We examine the ways in which deceptive tactics are used and endorsed in non-repeated human negotiation and show that prior experience plays a key role in governing what tactics are seen as acceptable or useful in negotiation. Previous work has indicated that people that negotiate through artificial agent representatives may be more inclined to fairness than those people that negotiate directly. We present a series of three user studies that challenge this initial assumption and expand on this picture by examining the role of past experience.
This work constructs a new scale for measuring endorsement of manipulative negotiation tactics and introduces its use to artificial intelligence research. It continues by presenting the results of a series of three studies that examine how negotiating experience can change what negotiation tactics and strategies human endorse. Study #1 looks at human endorsement of deceptive techniques based on prior negotiating experience as well as representative effects. Study #2 further characterizes the negativity of prior experience in relation to endorsement of deceptive techniques. Finally, in Study #3, we show that the lessons learned from the empirical observations in Study #1 and #2 can in fact be induced—by designing agents that provide a specific type of negative experience, human endorsement of deception can be predictably manipulated.
Toyoda, Y., Lucas, G. M., & Gratch, J. (2020). The effects of autonomy and task meaning in algorithmic management of crowdwork. In Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS 2020) (pp. 1404–1412). Auckland, New Zealand: International Foundation for Autonomous Agents and Multiagent Systems. https://doi.org/10.65109/anak9407 Show abstract
With the tremendous development of AI technologies, people will increasingly encounter software algorithms that supervise their work. Algorithmic management is the term for AI that performs the functions traditionally reserved for human managers (hiring, firing, providing evaluative feedback, and setting compensation). Although such algorithms indisputably perform management functions, they are often framed as support tools that facilitate worker autonomy. Perceptions of autonomy can enhance productivity, especially when the work holds intrinsic meaning for workers. But crowdwork often seems meaningless. More problematically, the meaning of the work must sometimes be obscured due to reasons of security or experimental control (when the workers serve as subjects in a psychological experiment). In this paper, we conduct an online experiment (N = 560) to investigate how autonomy-perceptions and the meaningfulness of work interact to shape crowdworker motivation. As predicted, we find that workers are motivated when their work has meaning and algorithmic management is framed in a way that makes worker autonomy salient. However, when work holds no meaning, we find productivity is enhanced when algorithms are framed in a way that makes algorithm control salient. We also find evidence that providing meaning to the work can introduce systematic biases in crowdworker responses that could undermine accuracy in certain contexts.
Keywords. Worker motivation; Crowdwork; Feedback system; Self determination theory
Bellas, A., Perrin, S., Malone, B., Rogers, K., Lucas, G. M., Phillips, E., Tossell, C., & de Visser, E. (2020). Rapport building with social robots as a method for improving mission debriefing in human-robot teams. In 2020 Systems and Information Engineering Design Symposium (SIEDS) (pp. 160–163). IEEE. https://doi.org/10.1109/SIEDS49339.2020.9106643 Show abstract
Conflicts may arise at any time during military debriefing meetings, especially in high intensity deployed settings. When such conflicts arise, it takes time to get everyone back into a receptive state of mind so that they engage in reflective discussion rather than unproductive arguing. It has been proposed by some that the use of social robots equipped with social abilities such as emotion regulation through rapport building may help to deescalate these situations to facilitate critical operational decisions. However, in military settings, the same AI agent used in the pre-brief of a mission may not be the same one used in the debrief. The purpose of this study was to determine whether a brief rapport-building session with a social robot could create a connection between a human and a robot agent, and whether consistency in the embodiment of the robot agent was necessary for maintaining this connection once formed. We report the results of a pilot study conducted at the United States Air Force Academy which simulated a military mission (i.e., Gravity and Strike). Participants’ connection with the agent, sense of trust, and overall likeability revealed that early rapport building can be beneficial for military missions.
Keywords. Artificial Intelligence, Conflict, Trust
de Gennaro, M., Krumhuber, E. G., & Lucas, G. M. (2020). Effectiveness of an empathic chatbot in combating adverse effects of social exclusion on mood. Frontiers in Psychology, 10, 3061. https://doi.org/10.3389/fpsyg.2019.03061 Show abstract
From past research it is well known that social exclusion has detrimental consequences for mental health. To deal with these adverse effects, socially excluded individuals frequently turn to other humans for emotional support. While chatbots can elicit social and emotional responses on the part of the human interlocutor, their effectiveness in the context of social exclusion has not been investigated. In the present study, we examined whether an empathic chatbot can serve as a buffer against the adverse effects of social ostracism. After experiencing exclusion on social media, participants were randomly assigned to either talk with an empathetic chatbot about it (e.g., "I'm sorry that this happened to you") or a control condition where their responses were merely acknowledged (e.g., "Thank you for your feedback"). Replicating previous research, results revealed that experiences of social exclusion dampened the mood of participants. Interacting with an empathetic chatbot, however, appeared to have a mitigating impact. In particular, participants in the chatbot intervention condition reported higher mood than those in the control condition. Theoretical, methodological, and practical implications, as well as directions for future research are discussed.
Keywords. chatbot, empathy, mood, social exclusion, virtual human
Fukumura, Y. E., Becerik-Gerber, B., Lucas, G. M., & Roll, S. C. (2020). Understanding office worker behavior to inform an artificial intelligence workstation [Poster presentation]. NIH Health in Buildings Roundtable Conference, Virtual Presentation.
Haring, K. S., Tobias, J., Waligora, J., Phillips, E., Tenhundfeld, N. L., Lucas, G. M., de Visser, E. J., Gratch, J., & Tossel, C. (2019). Conflict mediation in human-machine teaming: Using a virtual agent to support mission planning and debriefing. In 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN) (pp. 1–7). IEEE. https://doi.org/10.1109/RO-MAN46459.2019.8956414 Show abstract
Socially intelligent artificial agents and robots are anticipated to become ubiquitous in home, work, and military environments. With the addition of such agents to human teams it is crucial to evaluate their role in the planning, decision making, and conflict mediation processes. We conducted a study to evaluate the utility of a virtual agent that provided mission planning support in a three-person human team during a military strategic mission planning scenario. The team consisted of a human team lead who made the final decisions and three supporting roles, two humans and the artificial agent. The mission outcome was experimentally designed to fail and introduced a conflict between the human team members and the leader. This conflict was mediated by the artificial agent during the debriefing process through discuss or debate and open communication strategies of conflict resolution [1]. Our results showed that our teams experienced conflict. The teams also responded socially to the virtual agent, although they did not find the agent beneficial to the mediation process. Finally, teams collaborated well together and perceived task proficiency increased for team leaders. Socially intelligent agents show potential for conflict mediation, but need careful design and implementation to improve team processes and collaboration.
Khashe, S., Lucas, G. M., Becerik-Gerber, B., & Gratch, J. (2019). Establishing social dialog between buildings and their users. International Journal of Human–Computer Interaction, 35(17), 1545–1556. https://doi.org/10.1080/10447318.2018.1555346 Show abstract
Behavioral intervention strategies have yet to become successful in the development of initiatives to foster pro-environmental behaviors in buildings. In this paper, we explored the potentials of increasing the effectiveness of requests aiming to promote pro-environmental behaviors by engaging users in a social dialog, given the effects of two possible personas that are more related to the buildings (i.e., building vs. building manager). We tested our hypotheses and evaluated our findings in virtual and physical environments and found similar effects in both environments. Our results showed that social dialog involvement persuaded respondents to perform more pro-environmental actions. However, these effects were significant when the requests were delivered by an agent representing the building. In addition, these strategies were not equally effective across all types of people and their effects varied for people with different characteristics. Our findings provide useful design choices for persuasive technologies aiming to promote pro-environmental behaviors.
Lucas, G. M., Lehr, J., Krämer, N., & Gratch, J. (2019). The effectiveness of social influence tactics when used by a virtual agent. In Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents (pp. 22–29). Association for Computing Machinery. https://doi.org/10.1145/3308532.3329464 Show abstract
Research in social science distinguishes between two types of social influence: informational and normative. Informational social influence is driven by the desire to evaluate ambiguous situations correctly, whereas normative social influence is driven by the desire to be liked and gain social acceptance from another person. Although we know from research that humans can effectively use either of these techniques to persuade other humans, scholars have yet to examine the relative effectiveness of informational versus normative social influence when used by virtual agents. We report a study in which users interact with a system that persuades them either using informational or normative social influence. Furthermore, to compare agents to human interlocutors, users are told that the system is either tele-operated by a human (avatar) or fully-automated (agent). Using this design, we are able to compare the effectiveness of virtual agents (vs humans) in employing informational versus normative social influence. Participants interacted with the system, which employed a Wizard-of-Oz operated virtual agent that tried to persuade the user to agree with its rankings on a "survival task." Controlling for initial divergence in rankings between user and the agent, there was a significant main effect such that informational social influence resulted in greater influence than normative influence. However, this was qualified by an interaction that approached significance; users were, if anything, more persuaded by informational influence when they believe the agent was AI (compared to a human), whereas there was no difference between the agent and avatar in the normative influence condition.
Keywords. Virtual agents, social influence, persuasion, normative influence, informational influence
Johnson, E., Roediger, S., Lucas, G. M., & Gratch, J. (2019). Assessing common errors students make when negotiating. In Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents (pp. 30–37). New York, New York, USA: Association for Computing Machinery. https://doi.org/10.1145/3308532.3329470 Show abstract
Research has shown that virtual agents can be effective tools for teaching negotiation. Virtual agents provide an opportunity for students to practice their negotiation skills which leads to better outcomes. However, these negotiation training agents often lack the ability to understand the errors students make when negotiating, thus limiting their effectiveness as training tools. In this article, we argue that automated opponent-modeling techniques serve as effective methods for diagnosing important negotiation mistakes. To demonstrate this, we analyze a large number of participant traces generated while negotiating with a set of automated opponents. We show that negotiators' performance is closely tied to their understanding of an opponent's preferences. We further show that opponent modeling techniques can diagnose specific errors including: failure to elicit diagnostic information from an opponent, failure to utilize the information that was elicited, and failure to understand the transparency of an opponent. These results show that opponent modeling techniques can be effective methods for diagnosing and potentially correcting crucial negotiation errors.
Lee, M., Lucas, G. M., Mell, J., Johnson, E., & Gratch, J. (2019). What’s on your virtual mind? Mind perception in human-agent negotiations. In Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents (pp. 38–45). Association for Computing Machinery. https://doi.org/10.1145/3308532.3329465 Show abstract
Recent research shows that how we respond to other social actors depends on what sort of mind we ascribe to them. In this article we examine how perceptions of a virtual agent's mind shape behavior in human-agent negotiations. We varied descriptions and communicative behavior of virtual agents on two dimensions according to the mind perception theory: agency (cognitive aptitude) and patiency (affective aptitude). Participants then engaged in negotiations with the different agents. People scored more points and engaged in shorter negotiations with agents described to be cognitively intelligent, and got lower points and had longer negotiations with agents that were described to be cognitively unintelligent. Accordingly, agents described as having low agency ended up earning more points than those with high agency. Within the negotiations themselves, participants sent more happy and surprise emojis and emotionally valenced messages to agents described to be emotional. This high degree of described patiency also affected perceptions of the agent's moral standing and relatability. In short, manipulating the perceived mind of agents affects how people negotiate with them. We discuss these results, which show that agents are perceived not only as social actors, but as intentional actors through negotiations.
Keywords. Virtual agent, mind perception theory, theory of mind, human-agent negotiation, IAGO negotiation platform
Zhu, R., Becerik-Gerber, B., Lucas, G. M., Southers, E., & Pynadath, D. V. (2019). Information requirements for virtual environments to study human-building interactions during active shooter incidents. In Computing in Civil Engineering 2019 (pp. 188–195). Reston, VA: American Society of Civil Engineers. https://doi.org/10.1061/9780784482445.024 Show abstract
Active shooter incidents present an increasing American homeland security threat to public safety and human life. Several municipal law enforcement agencies have released building design guidelines intended to offer increased resilience and resistance to potential attacks. However, these design recommendations mainly focus on terrorist attacks, prioritizing the enhancement of building security, whereas their impact on safety during active shooter incidents, and corresponding human-building interactions (HBIs) that influence the outcomes (response performance), remain unclear. To respond to this research gap, virtual reality, with its ability to manipulate environmental variables and scenarios while providing safe non-invasive environments, could be a promising method to conduct human-subject studies in the context of active shooter incidents. In this paper, we identify the requirements for developing virtual environments that represent active shooter incidents in buildings to study HBIs and their impacts on the response performance. Key components constituting virtual environments were considered and presented. These include: (1) what types of buildings should be modeled in virtual environments; (2) how to select protective building design recommendations for active shooter incidents and model them in virtual environments; (3) what types of adversary and crowd behavior should be modeled; and (4) what types of interactions among participants, buildings, adversaries, and crowds should be included in virtual environments. Findings on the above key components were summarized to provide recommendations for future research directions.
Johnson, E., Lucas, G. M., Kim, P., & Gratch, J. (2019). Intelligent tutoring system for negotiation skills training. In Artificial Intelligence in Education: 20th International Conference, AIED 2019 (Lecture Notes in Computer Science, Vol. 11626) (pp. 122–127). Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-030-23207-8_23 Show abstract
Intelligent tutoring systems have proven very effective at teaching hard skills such as math and science, but less research has examined how to teach “soft” skills such as negotiation. In this paper, we introduce an effective approach to teaching negotiation tactics. Prior work showed that students can improve through practice with intelligent negotiation agents. We extend this work by proposing general methods of assessment and feedback that could be applied to a variety of such agents. We evaluate these techniques through a human subject study. Our study demonstrates that personalized feedback improves students’ use of several foundational tactics.
Keywords. Negotiation training, Individualized feedback, Soft skills training
Aryal, A., Becerik-Gerber, B., Anselmo, F., Roll, S. C., & Lucas, G. (2019). Smart desks to promote comfort, health and productivity in offices: A vision for future workplaces. Frontiers in Built Environment, 5, 76. https://doi.org/10.3389/fbuil.2019.00076 Show abstract
People spend most of their day in buildings, and a large portion of the energy in buildings is used to control the indoor environment for creating acceptable conditions for occupants. However, majority of the building systems are controlled based on a ‘one size fits all’ scheme which cannot account for individual occupant preferences. This leads to discomfort, low satisfaction and negative impacts on occupants’ productivity, health and well-being. In this paper, we describe our vision of how recent advances in Internet of Things (IoT) and machine learning can be used to add intelligence to an office desk to personalize the environment around the user. The smart desk can learn individual user preferences for the indoor environment, personalize the environment based on user preferences, and act as an intelligent support system for improving user comfort, health and productivity. We briefly describe the recent advances made in different domains that can be leveraged to enhance occupant experience in buildings and describe the overall framework for the smart desk. We conclude the paper with a discussion of possible avenues for further research.
Keywords: Personalized environments, Smart desks, IoT (internet of things), Smart buildings, Indoor environment quality (IEQ)
Chu, V. C., Lucas, G. M., Lei, S., Mozgai, S., Khooshabeh, P., & Gratch, J. (2019). Emotion regulation in the prisoner’s dilemma: Effects of reappraisal on behavioral measures and cardiovascular measures of challenge and threat. Frontiers in Human Neuroscience, 13, 50. https://doi.org/10.3389/fnhum.2019.00050 Show abstract
The current study examines cooperation and cardiovascular responses in individuals that were defected on by their opponent in the first round of an iterated Prisoner's Dilemma. In this scenario, participants were either primed with the emotion regulation strategy of reappraisal or no emotion regulation strategy, and their opponent either expressed an amused smile or a polite smile after the results were presented. We found that cooperation behavior decreased in the no emotion regulation group when the opponent expressed an amused smile compared to a polite smile. In the cardiovascular measures, we found significant differences between the emotion regulation conditions using the biopsychosocial (BPS) model of challenge and threat. However, the cardiovascular measures of participants instructed with the reappraisal strategy were only weakly comparable with a threat state of the BPS model, which involves decreased blood flow and perception of greater task demands than resources to cope with those demands. Conversely, the cardiovascular measures of participants without an emotion regulation were only weakly comparable with a challenge state of the BPS model, which involves increased blood flow and perception of having enough or more resources to cope with task demands.
Keywords. Prisoner’s Dilemma, biopsychosocial model, facial expression, physiological data, reappraisal
Lucas, G. M., Rizzo, A., Gratch, J., Scherer, S., Stratou, G., Boberg, J., & Morency, L.-P. (2019). Reporting mental health symptoms: Breaking down barriers to care with virtual human interviewers. In M. Slater, M. V. Sanchez-Vives, A. Rizzo, & M. Bergamasco (Eds.), The impact of virtual and augmented reality on individuals and society (pp. 256–264). Lausanne, Switzerland: Frontiers Media. https://doi.org/10.3389/978-2-88963-064-6 Show abstract
A common barrier to healthcare for psychiatric conditions is the stigma associated with these disorders. Perceived stigma prevents many from reporting their symptoms. Stigma is a particularly pervasive problem among military service members, preventing them from reporting symptoms of combat-related conditions like posttraumatic stress disorder (PTSD). However, research shows increased reporting by service members when anonymous assessments are used. For example, service members report more symptoms of PTSD when they anonymously answer the Post-Deployment Health Assessment (PDHA) symptom checklist compared to the official PDHA, which is identifiable and linked to their military records. To investigate the factors that influence reporting of psychological symptoms by service members, we used a transformative technology: automated virtual humans that interview people about their symptoms. Such virtual human interviewers allow simultaneous use of two techniques for eliciting disclosure that would otherwise be incompatible; they afford anonymity while also building rapport. We examined whether virtual human interviewers could increase disclosure of mental health symptoms among active-duty service members that just returned from a year-long deployment in Afghanistan. Service members reported more symptoms during a conversation with a virtual human interviewer than on the official PDHA. They also reported more to a virtual human interviewer than on an anonymized PDHA. A second, larger sample of active-duty and former service members found a similar effect that approached statistical significance. Because respondents in both studies shared more with virtual human interviewers than an anonymized PDHA—even though both conditions control for stigma and ramifications for service members’ military records—virtual human interviewers that build rapport may provide a superior option to encourage reporting.
Keywords. virtual humans, assessment, disclosure, psychological symptoms, anonymity
Lucas, G. M., Krämer, N., Peters, C., Taesch, L.-S., Mell, J., & Gratch, J. (2018). Effects of perceived agency and message tone in responding to a virtual personal trainer. In Proceedings of the 18th International Conference on Intelligent Virtual Agents (pp. 247–254). Association for Computing Machinery. https://doi.org/10.1145/3267851.3267855 Show abstract
Research has demonstrated promising benefits of applying virtual trainers to promote physical fitness. The current study investigated the value of virtual agents in the context of personal fitness, compared to trainers with greater levels of perceived agency (avatar or live human). We also explored the possibility that the effectiveness of the virtual trainer might depend on the affective tone it uses when trying to motivate users. Accordingly, participants received either positively or negatively valenced motivational messages from a virtual human they believed to be either an agent or an avatar, or they received the messages from a human instructor via skype. Both self-report and physiological data were collected. Like in-person coaches, the live human trainer who used negatively valenced messages were well-regarded; however, when the agent or avatar used negatively valenced messages, participants responded more poorly than when they used positively valenced ones. Perceived agency also affected rapport: compared to the agent, users felt more rapport with the live human trainer or the avatar. Regardless of trainer type, they also felt more rapport - and said they put in more effort - with trainers that used positively valenced messages than those that used negatively valenced ones. However, in reality, they put in more physical effort (as measured by heart rate) when trainers employed the more negatively valenced affective tone. We discuss implications for human–computer interaction.
Keywords. Social agents, physical fitness, affective tone, rapport, persuasion
Hoegen, R., van der Schalk, J., Lucas, G. M., & Gratch, J. (2018). The impact of agent facial mimicry on social behavior in a prisoner’s dilemma. In Proceedings of the 18th International Conference on Intelligent Virtual Agents (pp. 275–280). Association for Computing Machinery. https://doi.org/10.1145/3267851.3267911 Show abstract
A long tradition of research suggests a relationship between emotional mimicry and pro-social behavior, but the nature of this relationship is unclear. Does mimicry cause rapport and cooperation, or merely reflect it? Virtual humans can provide unique insights into these social processes by allowing unprecedented levels of experimental control. In a 2 x 2 factorial design, we examined the impact of facial mimicry and counter-mimicry in the iterated prisoner's dilemma. Participants played with an agent that copied their smiles and frowns or one that showed the opposite pattern -- i.e., that frowned when they smiled. As people tend to smile more than frown, we independently manipulated the contingency of expressions to ensure any effects are due to mimicry alone, and not the overall positivity/negativity of the agent: i.e., participants saw either a reflection of their own expressions or saw the expressions shown to a previous participant. Results show that participants smiled significantly more when playing an agent that mimicked them. Results also show a complex association between smiling, feelings of rapport, and cooperation. We discuss the implications of these findings on virtual human systems and theories of cooperation.
Keywords. Virtual Humans, Facial mimicry, Emotion recognition
Mell, J., Lucas, G. M., Mozgai, S., Boberg, J., Artstein, R., & Gratch, J. (2018). Towards a repeated negotiating agent that treats people individually: Cooperation, social value orientation, & Machiavellianism. In Proceedings of the 18th International Conference on Intelligent Virtual Agents (pp. 125–132). Association for Computing Machinery. https://doi.org/10.1145/3267851.3267910 Show abstract
We present the results of a study in which humans negotiate with computerized agents employing varied tactics over a repeated number of economic ultimatum games. We report that certain agents are highly effective against particular classes of humans: several individual difference measures for the human participant are shown to be critical in determining which agents will be successful. Asking for favors works when playing with pro-social people but backfires with more selfish individuals. Further, making poor offers invites punishment from Machiavellian individuals. These factors may be learned once and applied over repeated negotiations, which means user modeling techniques that can detect these differences accurately will be more successful than those that don't. Our work additionally shows that a significant benefit of cooperation is also present in repeated games—after sufficient interaction. These results have deep significance to agent designers who wish to design agents that are effective in negotiating with a broad swath of real human opponents. Furthermore, it demonstrates the effectiveness of techniques which can reason about negotiation over time.
Keywords. Human-Agent Negotiation, Personality Measures
Lucas, G. M., Boberg, J., Traum, D., Artstein, R., Gratch, J., Gainer, A., Johnson, E., Leuski, A., & Nakano, M. (2018). Culture, errors, and rapport-building dialogue in social agents. In Proceedings of the 18th International Conference on Intelligent Virtual Agents (pp. 51–58). Association for Computing Machinery. https://doi.org/10.1145/3267851.3267887 Show abstract
This work explores whether culture impacts the extent to which social dialogue can mitigate (or exacerbate) the loss of trust caused when agents make conversational errors. Our study uses an agent designed to persuade users to agree with its rankings on two tasks. Participants from the U.S. and Japan completed our study. We perform two manipulations: (1) The presence of conversational errors – the agent exhibited errors in the second task or not; (2) The presence of social dialogue – between the two tasks, users either engaged in a social dialogue with the agent or completed a control task. Replicating previous research, conversational errors reduce the agent's influence. However, we found that culture matters: there was a marginally significant three-way interaction with culture, presence of social dialogue, and presence of errors. The pattern of results suggests that, for American participants, social dialogue backfired if it is followed by errors, presumably because it extends the period of good performance, creating a stronger contrast effect with the subsequent errors. However, for Japanese participants, social dialogue if anything mitigates the detrimental effect of errors; the negative effect of errors is only seen in the absence of a social dialogue. Agent design should therefore take the culture of the intended users into consideration when considering use of social dialogue to bolster agents against conversational errors.
Keywords. Social agents, influence, social dialogue, rapport, conversational errors, culture
Bennett, J. B., Lucas, G. M., Linde, B. D., Neeper, M. A., Hudson, M., & Gatchel, R. J. (2018). A process model of health consciousness: Its application to the prevention of workplace prescription drug misuse. Journal of Applied Biobehavioral Research, 23(3), e12130. https://doi.org/10.1111/jabr.12130 Show abstract
The concept of health consciousness (HC), previously studied as a dispositional construct, holds promise for brief prevention training in work settings. The current study developed a more process-oriented model of HC. Employer’s increasing concerns about worker prescription drug (Rx) misuse represents an opportunity to study and improve HC. Two pilot studies delivered a brief training program and assessed short-term, pre- to post–self-reports of factors related to HC. The first pilot (n = 162) attempted to evaluate a stress/resilience program (“Raw Coping Power”) that sought to improve worker ability to recognize and correct unhealthy coping behaviors. A second pilot (n = 114) attempted to evaluate an Rx misuse prevention program that sought to enhance awareness of HC as a protective factor. Both pilots used a driver analysis to examine correlates of improved HC. These correlates included increased confidence in one’s ability to evaluate risks, and awareness of healthy alternatives. Many employees also found the concept of HC useful. Previous writings about HC, in combination with current results, informed the design of a new, process-oriented model of HC. The current paper offers this model and a driver analysis methodology that future researchers can use to further explore and more fully evaluate brief HC interventions in work settings.
Mell, J., Lucas, G. M., & Gratch, J. (2018). Welcome to the real world: How agent strategy increases human willingness to deceive. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS 2018) (pp. 1250–1257). International Foundation for Autonomous Agents and Multiagent Systems. https://doi.org/10.65109/wugf4923 Show abstract
Humans that negotiate through representatives often instruct those representatives to act in certain ways that align with both the client’s goals and his or her social norms. However, which tactics and ethical norms humans endorse vary widely from person to person, and these endorsements may be easy to manipulate. This work presents the results of a study that demonstrates that humans that interact with an artificial agent may change what kinds of tactics and norms they endorse—often dramatically. Previous work has indicated that people that negotiate through artificial agent representatives may be more inclined to fairness than those people that negotiate directly. Our work qualifies that initial picture, demonstrating that subsequent experience may change this tendency toward fairness. By exposing human negotiators to tough, automated agents, we are able to shift the participant’s willingness to deceive others and utilize “hard-ball” negotiation techniques. In short, what techniques people decide to endorse is dependent upon their context and experience.
We examine the effects of interacting with four different types of automated agents, each with a unique strategy, and how this subsequently changes which strategies a human negotiator might later endorse. In the study, which was conducted on an online negotiation platform, four different types of automated agents negotiate with humans over the course of a 10-minute interaction. The agents differ in a 2x2 design according to agent strategy (tough vs. fair) and agent attitude (nice vs. nasty). These results show that in this multi-issue bargaining task, humans that interacted with a tough agent were more willing to endorse deceptive techniques when instructing their own representative. These kinds of techniques were endorsed even if the agent the human encountered did not use deception as part of its strategy. In contrast to some previous work, there was not a significant effect of agent attitude. These results indicate the power of allowing people to program agents that follow their instructions, but also indicate that these social norms and tactic endorsements may be mutable in the presence of real negotiation experience.
Keywords. Human-computer interaction, empirical studies, negotiation, IAGO Negotiation platform, social norms, socially-aware agents
Khashe, S., Becerik-Gerber, B., Lucas, G. M., & Gratch, J. (2018). Persuasive effects of immersion in virtual environments for measuring pro-environmental behaviors. In Proceedings of the 35th International Symposium on Automation and Robotics in Construction (ISARC 2018) (pp. 1205–1211). International Association for Automation and Robotics in Construction. https://doi.org/10.22260/ISARC2018/0167 Show abstract
In this study, we investigated the effects of immersive vs. non-immersive virtual environment (VE) platforms (i.e., head mounted display (HMD) vs. laptop PC) on compliance with pro-environmental behaviors. We performed a detailed analysis of the effects of these VE platforms on other variables, such as task performance, sense of presence, and simulator sickness. We also explored the factors, such as participants’ gender and immersive tendency that could have influenced the effects of VE platforms. In a between subject design, 100 participants were randomly assigned to interact with either a desktop or an HMD. The results showed no significant effects of VE platforms on compliance with proenvironmental requests, task performance, and sense of presence. However, the HMD elicited higher simulator sickness compared to the desktop display. In addition, we demonstrated there was a strong relationship between participants’ immersive tendency and the presence that they experienced. Our findings provide empirical evidence capable of helping researchers select an appropriate VE platform when investigating the influence of behavioral interventions aim to promote sustainable behaviors.
Keywords. Virtual environments, pro-environmental behavior, immersion, presence, simulator sickness, compliance
Monahan, S., Johnson, E., Lucas, G. M., Finch, J., & Gratch, J. (2018). Autonomous agent that provides automated feedback improves negotiation skills. In Artificial Intelligence in Education: 19th International Conference, AIED 2018 (Lecture Notes in Computer Science, Vol. 10948) (pp. 225–229). Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-319-93846-2_41 Show abstract
Research has found that individuals can improve their negotiation abilities by practicing with virtual agents [1, 2]. For these pedagogical agents to become more “intelligent,” the system should be able to give feedback on negotiation performance [3, 4]. In this study, we examined the impact of providing such individualized feedback. Participants first engaged in a negotiation with a virtual agent. After this negotiation, participants were either given automated individualized feedback or not. Feedback was based on negotiation principles [4], which were quantified using a validated approach [5]. Participants then completed a second, parallel negotiation. Our results show that, compared to the control condition, participants who received such feedback after the first negotiation showed a significantly greater improvement in the strength of their first offer, concession curve, and thus their final outcome in the negotiation.
Keywords. Negotiation, Individualized feedback, Automated metrics
Bennett, J. B., Neeper, M., Linde, B. D., Lucas, G. M., & Simone, L. (2018). Team resilience training in the workplace: E-learning adaptation, measurement model, and two pilot studies. JMIR Mental Health, 5(2), e35. https://doi.org/10.2196/mental.8955 Show abstract
Background. The majority of resilience interventions focus on the individual. Workplace resilience is a growing field of research. Given the ever-increasing interconnectedness in businesses, teamwork is a guarantee. There is also growing recognition that resilience functions at the team level.
Objective. The objective of our work was to address three shortcomings in the study of workplace resilience interventions: lack of interventions focusing on group-level or team resilience, the need for brief interventions, and the need for more theoretical precision in intervention studies.
Methods. The authors took an established evidence-based program (Team Resilience) and modified it based on these needs. A working model for brief intervention evaluation distinguishes outcomes that are proximal (perceptions that the program improved resilience) and distal (dispositional resilience). A total of 7 hypotheses tested the model and program efficacy.
Results. Two samples (n = 118 and n = 181) of engineering firms received the Web-based training and provided immediate reactions in a posttest-only design. The second sample also included a control condition (n = 201). The findings support the model and program efficacy. For example, workplace resilience was greater in the intervention group than in the control group. Other findings suggest social dissemination effects, equal outcomes for employees at different stress levels, and greater benefit for females.
Conclusions. This preliminary research provides evidence for the capabilities of e-learning modules to effectively promote workplace resilience and a working model of team resilience.
Keywords. early intervention, experimental design, gender, incentives, online learning, psychological theory, quasi-experimental, questionnaire design, resilience, social support, stress, workplace
Artstein, R., Boberg, J., Gainer, A., Gratch, J., Johnson, E., Leuski, A., Lucas, G. M., & Traum, D. (2018). The Niki and Julie corpus: Collaborative multimodal dialogues between humans, robots, and virtual agents. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). European Language Resources Association. https://doi.org/10.63317/5m3sw35yfkrx Show abstract
The Niki and Julie corpus contains more than 600 dialogues between human participants and a human-controlled robot or virtual agent, engaged in a series of collaborative item-ranking tasks designed to measure influence. Some of the dialogues contain deliberate conversational errors by the robot, designed to simulate the kinds of conversational breakdown that are typical of present-day automated agents. Data collected include audio and video recordings, the results of the ranking tasks, and questionnaire responses; some of the recordings have been transcribed and annotated for verbal and nonverbal feedback. The corpus has been used to study influence and grounding in dialogue. All the dialogues are in American English.
Keywords. dialogue, human-robot interaction, collaborative problem-solving, social influence
Pincus, E., Lei, S., Lucas, G. M., Johnson, E., Tsang, M., Gratch, J., & Traum, D. (2018). The importance of regulatory fit & early success in a human-machine game. In Proceedings of the Technology, Mind, and Society (pp. 1–6). Association for Computing Machinery. https://doi.org/10.1145/3183654.3183661 Show abstract
In this paper, we explore the potential of regulatory focus theory as a framework for personalizing human-machine interactions. We manipulate framing (gain or loss) of a collaborative word-guessing game where a fully-automated virtual human gives clues. Consistent with previous work on regulatory focus, we find evidence of significantly higher perceived task-success when participants have regulatory fit. Inconsistent with previous work, however, fit did not increase task-enjoyment (nor performance). Participants with gain framing had marginally higher enjoyment, regardless of their regulatory focus. We operationalize motivation by number of optional rounds played but failed to find a "fit" effect. Instead, players who achieved early success (scoring more points in initial rounds) were more motivated. Early success was significantly correlated with number of optional rounds played. This finding calls to attention the need for the literature to more thoroughly investigate the relationship between success-timing and total player playtime in the game.
Keywords. Regulatory Focus Theory, Regulatory-Fit, Word-Guessing Games, Clues, Guesses, Virtual Human, Embodiment, Promotion Orientation, Prevention Orientation, Gain, Loss
Wauck, H., Lucas, G. M., Shapiro, A., Feng, A., Boberg, J., & Gratch, J. (2018). Analyzing the effect of avatar self-similarity on men and women in a search and rescue game. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (pp. 1–12). Association for Computing Machinery. https://doi.org/10.1145/3173574.3174059 Show abstract
A crucial aspect of virtual gaming experiences is the avatar: the player's virtual self-representation. While research has demonstrated benefits to using self-similar avatars in some virtual experiences, such avatars sometimes produce a more negative experience for women. To help researchers and game designers assess the cost-benefit tradeoffs of self-similar avatars, we compared players' performance and subjective experience in a search and rescue computer game when using two different photorealistic avatars: their own self or a friend, and when playing either a social (rescuing people) or a nonsocial (rescuing gems) version of the game. There was no effect of avatar appearance on players' performance or subjective experience in either game version, but we also found that women's experience with self-similar avatars was no more negative than men's. Our results suggest that avatar appearance may not make a difference to players in certain game contexts.
Keywords. avatars, photorealism, identification, games, gender, player experience, performance
Khooshabeh, P., & Lucas, G. M. (2018). Virtual human role players for studying social factors in organizational decision making. Frontiers in Psychology, 9, 194. https://doi.org/10.3389/fpsyg.2018.00194 Show abstract
The cyber domain of military operations presents many challenges. A unique element is the social dynamic between cyber operators and their leadership because of the novel subject matter expertise involved in conducting technical cyber tasks, so there will be situations where senior leaders might have much less domain knowledge or no experience at all relative to the warfighters who report to them. Nonetheless, it will be important for junior cyber operators to convey convincing information relevant to a mission in order to persuade or influence a leader to make informed decisions. The power dynamic will make it difficult for the junior cyber operator to successfully influence a higher ranking leader. Here we present a perspective with a sketch for research paradigm(s) to study how different factors (normative vs. informational social influence, degree of transparency, and perceived appropriateness of making suggestions) might interact with differential social power dynamics of individuals in cyber decision-making contexts. Finally, we contextualize this theoretical perspective for the research paradigms in viable training technologies.
Keywords. emotion, persuasion, social influence, training, virtual humans
Lucas, G. M., Boberg, J., Traum, D., Artstein, R., Gratch, J., Gainer, A., Johnson, E., Leuski, A., & Nakano, M. (2018). Getting to know each other: The role of social dialogue in recovery from errors in social robots. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction (pp. 344–351). Association for Computing Machinery. https://doi.org/10.1145/3171221.3171258 Show abstract
This work explores the extent to which social dialogue can mitigate (or exacerbate) the loss of trust caused when robots make conversational errors. Our study uses a NAO robot programmed to persuade users to agree with its rankings on two tasks. We perform two manipulations: (1) The timing of conversational errors - the robot exhibited errors either in the first task, the second task, or neither; (2) The presence of social dialogue - between the two tasks, users either engaged in a social dialogue with the robot or completed a control task. We found that the timing of the errors matters: replicating previous research, conversational errors reduce the robot's influence in the second task, but not on the first task. Social dialogue interacts with the timing of errors, acting as an intensifier: social dialogue helps the robot recover from prior errors, and actually boosts subsequent influence; but social dialogue backfires if it is followed by errors, because it extends the period of good performance, creating a stronger contrast effect with the subsequent errors. The design of social robots should therefore be more careful to avoid errors after periods of good performance than early on in a dialogue.
Keywords. Social robots, influence, social dialogue, rapport, errors
Krämer, N. C., Lucas, G. M., Schmitt, L., & Gratch, J. (2018). Social snacking with a virtual agent – On the interrelation of need to belong and effects of social responsiveness when interacting with artificial entities. International Journal of Human-Computer Studies, 109, 112–121. https://doi.org/10.1016/j.ijhcs.2017.09.001 Show abstract
Based on considerations that people´s need to belong can be temporarily satisfied by “social snacking” (Gardner et al., 2005) in the sense that in absence of social interactions which adequately satisfy belongingness needs surrogates can bridge lonely times, it was tested whether the interaction with a virtual agent can serve to ease the need for social contact. In a between subjects experimental setting, 79 participants interacted with a virtual agent who either displayed socially responsive nonverbal behavior or not. Results demonstrate that although there was no main effect of socially responsive behavior on participants´ subjective experience of rapport and on connectedness with the agent, those people with a high need to belong reported less willingness to engage in social activities after the interaction with a virtual agent – but only if the agent displayed socially responsive behavior.
Keywords. Virtual agents, Social snacking, Need to belong, Socially responsive nonverbal behavior, Rapport
Aryal, A., Becerik-Gerber, B., Anselmo, F., Roll, S. C., & Lucas, G. M. (2018). Smart desk to promote health and productivity [Poster presentation]. NIH/NSF/CDC/GSA Health in Buildings for Today and Tomorrow: An Interdisciplinary Conference on Health and the Sustainable Built Environment, Bethesda, MD.
Lucas, G. M., Rizzo, A., Gratch, J., Scherer, S., Stratou, G., Boberg, J., & Morency, L.-P. (2017). Reporting mental health symptoms: Breaking down barriers to care with virtual human interviewers. Frontiers in Robotics and AI, 4, 51. https://doi.org/10.3389/frobt.2017.00051 Show abstract
A common barrier to healthcare for psychiatric conditions is the stigma associated with these disorders. Perceived stigma prevents many from reporting their symptoms. Stigma is a particularly pervasive problem among military service members, preventing them from reporting symptoms of combat-related conditions like posttraumatic stress disorder (PTSD). However, research shows increased reporting by service members when anonymous assessments are used. For example, service members report more symptoms of PTSD when they anonymously answer the Post-Deployment Health Assessment (PDHA) symptom checklist compared to the official PDHA, which is identifiable and linked to their military records. To investigate the factors that influence reporting of psychological symptoms by service members, we used a transformative technology: automated virtual humans that interview people about their symptoms. Such virtual human interviewers allow simultaneous use of two techniques for eliciting disclosure that would otherwise be incompatible; they afford anonymity while also building rapport. We examined whether virtual human interviewers could increase disclosure of mental health symptoms among active-duty service members that just returned from a year-long deployment in Afghanistan. Service members reported more symptoms during a conversation with a virtual human interviewer than on the official PDHA. They also reported more to a virtual human interviewer than on an anonymized PDHA. A second, larger sample of active-duty and former service members found a similar effect that approached statistical significance. Because respondents in both studies shared more with virtual human interviewers than an anonymized PDHA—even though both conditions control for stigma and ramifications for service members’ military records—virtual human interviewers that build rapport may provide a superior option to encourage reporting.
Keywords. virtual humans, assessment, Disclosure, Psychological symptoms, anonymity
Lucas, G. M., Boberg, J., Traum, D., Artstein, R., Gratch, J., Gainer, A., Johnson, E., Leuski, A., & Nakano, M. (2017). The role of social dialogue and errors in robots. In Proceedings of the 5th International Conference on Human Agent Interaction (pp. 431–433). Association for Computing Machinery. https://doi.org/10.1145/3125739.3132617 Show abstract
Social robots establish rapport with human users. This work explores the extent to which rapport-building can benefit (or harm) conversations with robots, and under what circumstances this occurs. For example, previous work has shown that agents that make conversational errors are less capable of influencing people than agents that do not make errors [1]. Some work has shown this effect with robots, but prior research has not considered additional factors such as the level of rapport between the person and the robot. We predicted that building rapport through a social dialogue (such as an ice-breaker) could mitigate the detrimental effect of a robot's errors on influence. Our study used a Nao robot programmed to persuade users to agree with its rankings on two "survival tasks" (e.g., lunar survival task). We manipulated both errors and social dialogue: the robot either exhibited errors in the second survival task or not, and users either engaged in an ice-breaker with the robot between the two survival tasks or completed a control task. Replicating previous research, errors tended to reduce the robot's influence in the second survival task. Contrary to our prediction, results revealed that the ice-breaker did not mitigate the effect of errors, and if anything, errors were more harmful after the ice-breaker (intended to build rapport) than in the control condition. This backfiring of attempted rapport-building may be due to a contrast effect, suggesting that the design of social robots should avoid introducing dialogues of incongruent quality.
Keywords. Social robots, influence, social dialogue, rapport, errors
Stratou, G., Hoegen, R., Lucas, G. M., & Gratch, J. (2017). Investigating gender differences in temporal dynamics during an iterated social dilemma: An automatic analysis using networks. In 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII) (pp. 531–536). IEEE. https://doi.org/10.1109/ACII.2017.8273650 Show abstract
Emotions have a temporal nature and very often personality traits and underlying psychological conditions are hidden in the dynamics of those expressions. Within this work, we investigate the dynamics of the facial displays of dyads during an iterated social dilemma. We focus on the effect of gender and gender-pairing on those behaviors. We use networks to capture the temporal dynamics and create measures of inter- and intra-personal dependencies of emotional states. Our analysis on an iterated prisoner's dilemma corpus suggests that there are gender differences on the transitions of the emotional states and the degree of emotional influence from the opponent.
Khashe, S., Lucas, G. M., Becerik-Gerber, B., & Gratch, J. (2017). Buildings with persona: Towards effective building-occupant communication. Computers in Human Behavior, 75, 607–618. https://doi.org/10.1016/j.chb.2017.05.040 Show abstract
Occupant behavior is one of the most significant contributors to building energy consumption. Employing communication systems to enable buildings to interact with their occupants and influence the way they behave could significantly reduce energy consumption. We investigated the effectiveness of different delivery styles (i.e., avatar, voice, and text), as well as the impact of communicator’s persona (i.e., building facility manager and building itself) and gender (i.e., male and female) on occupants’ compliance with pro-environmental requests. The results showed that avatar is more effective than voice and voice is more effective than text on promoting compliance with persuasive pro-environmental requests. In addition, results showed greater compliance with requests made by the persona of a building facility manager than the persona of the building itself. Finally, participants were more likely to comply with the female communicator than the male communicator. Accordingly, this new interaction between buildings and their occupants could impact human behavior.
Keywords. Compliance, Pro-environmental, Immersive virtual environment, Delivery styles, Persona, Gender
Lucas, G. M., Gratch, J., Malandrakis, N., Szablowski, E., Fessler, E., & Nichols, J. (2017). GOAALLL!: Using sentiment in the World Cup to explore theories of emotion. Image and Vision Computing, 65, 58–65. https://doi.org/10.1016/j.imavis.2017.01.006 Show abstract
Sporting events evoke strong emotions amongst fans and thus act as natural laboratories to explore emotions and how they unfold in the wild. Computational tools, such as sentiment analysis, provide new ways to examine such dynamic emotional processes. In this article we use sentiment analysis to examine tweets posted during 2014 World Cup. Such analysis gives insight into how people respond to highly emotional events, and how these emotions are shaped by contextual factors, such as prior expectations, and how these emotions change as events unfold over time. Here we report on some preliminary analysis of a World Cup twitter corpus using sentiment analysis techniques. We show these tools can give new insights into existing theories of what makes a sporting match exciting. This analysis seems to suggest that, contrary to assumptions in sports economics, excitement relates to expressions of negative emotion. We also discuss some challenges that such data present for existing sentiment analysis techniques and discuss future analysis.
Keywords. Sentiment; Sentiment analysis; Sport; Emotion models; Twitter
Mell, J., Lucas, G. M., & Gratch, J. (2017). Prestige questions, online agents, and gender-driven differences in disclosure. In Intelligent Virtual Agents: 17th International Conference, IVA 2017 (Lecture Notes in Computer Science, Vol. 10498) (pp. 273–282). Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-319-67401-8_36 Show abstract
This work considers the possibility of using virtual agents to encourage disclosure for sensitive information. In particular, this research used “prestige questions”, which asked participants to disclose information relevant to their socioeconomic status, such as credit limit, as well as university attendance, and mortgage or rent payments they could afford. We explored the potential for agents to enhance disclosure compared to conventional web-forms, due to their ability to serve as relational agents by creating rapport. To consider this possibility, agents were framed as artificially intelligent versus avatars controlled by a real human, and we compared these conditions to a version of the financial questionnaire with no agent. In this way, both the perceived agency of the agent and its ability to generate rapport were tested. Additionally, we examined the differences in disclosure between men and women in these conditions. Analyses reveled that agents (either AI- or human-framed) evoked greater disclosure compared to the no agent condition. However, there was some evidence that human-framed agents evoked greater lying. Thus, users in general responded more socially to the presence of a human- or AI-framed agent, and the benefits and costs of this approach were made apparent. The results are discussed in terms of rapport and anonymity.
Keywords. Virtual agents, Human-Agent experimentation, Disclosure
Mozgai, S., Lucas, G. M., & Gratch, J. (2017). To tell the truth: Virtual agents and morning morality. In Intelligent Virtual Agents: 17th International Conference, IVA 2017 (Lecture Notes in Computer Science, Vol. 10498) (pp. 283–286). Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-319-67401-8_37 Show abstract
This paper investigates the impact of time of day on truthfulness in human-agent interactions. Time of day has been found to have important implications for moral behavior in human-human interaction. Namely, the morning morality effect shows that people are more likely to act ethically (i.e., tell fewer lies) in the morning than in the afternoon. Based on previous work on disclosure and virtual agents, we propose that this effect will not bear out in human-agent interactions. Preliminary evaluation shows that individuals who lie when engaged in multi-issue bargaining tasks with the Conflict Resolution Agent, a semi-automated virtual human, tell more lies to human negotiation partners than virtual agent negotiation partners in the afternoon and are more likely to tell more lies in the afternoon than in the morning when they believe they are negotiating with a human. Time of day does not have a significant effect on the amount of lies told to the virtual agent during the multi-issue bargaining task.
Keywords. Morning morality, Virtual humans, Honest responding, Multi-issue bargaining
Nazari, Z., Lucas, G. M., & Gratch, J. (2017). Fixed-pie lie in action. In Intelligent Virtual Agents: 17th International Conference, IVA 2017 (Lecture Notes in Computer Science, Vol. 10498) (pp. 287–300). Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-319-67401-8_38 Show abstract
Negotiation is a crucial skill for socially intelligent agents. Sometimes negotiators lie to gain advantage. In particular, they can claim that they want the same thing as their opponents (i.e., use a “fixed-pie lie”) to gain an advantage while appearing fair. The current work is the first attempt to examine effectiveness of this strategy when used by agents against humans in realistic negotiation settings. Using the IAGO platform, we show that the exploitative agent indeed wins more points while appearing fair and honest to its opponent. In a second study, we investigated how far the exploitative agents could push for more gain and examined their effect on people’s behavior. This study shows that even though exploitative agents gained high value in short-term, their long-term success remains questioned as they left their opponents unhappy and unsatisfied.
Keywords. Human-agent negotiation, Behavioral game theory, Deception
Lucas, G. M., Neeper, M., Linde, B., & Bennett, J. (2017). Preventing prescription drug misuse in work settings: Efficacy of a brief intervention in health consciousness. Journal of Medical Internet Research, 19(7), e242. https://doi.org/10.2196/jmir.7828 Show abstract
Background. It is becoming more commonplace for employees to use prescription medication outside of intended use. Opioid and other prescription misuse has implications for the health and productivity of workers. Easy-to-access webinars that help employees learn about alternatives to prescription use may decrease risk.
Objective. The aim of this study was to examine the efficacy of an interactive but brief health consciousness and prescription drug intervention for a diverse sample of employees and show effectiveness via both Internet-delivered webinar and classroom delivery.
Methods. Employees from a variety of workplaces filled out pre- and post-questionnaires upon completion of a one-hour long intervention.
Results. A total of 114 participants completed the pre- and post-questionnaires. Results showed that, compared with before the training, participants reported significantly more knowledge about prescription drug misuse and alternatives to prescription drug use after the training (t113=7.91, P<.001). Moreover, the medium of presentation (ie, face-to-face vs webinar) did not significantly impact effectiveness of the training (F1,98=1.15, P=.29).
Conclusions. In both webinar and classroom formats, participants gained knowledge about alternatives to prescription drug use. This intervention appears to be beneficial to employees and assists in the awareness of prescription drug use in general and in the workplace.
Keywords. consciousness, education, health, prescription drugs, substance-related disorders, workplace
Bennett, J. B., Lucas, G. M., & Neeper, M. A. (2016, October). Preventing worker prescription drug misuse: Time for a new integrated approach. EASNA Research Notes, 6(1), 1–6. Employee Assistance Society of North America (EASNA). Full text Show abstract
Innovative approaches are needed to combat the risks of adult prescription drug misuse and abuse. Most employees are aware of the double-edged sword of prescription technologies. The offer of relief, disease management, enhanced performance, and improved mood are intertwined with a myriad of damaging side-effects as well as the potential for misuse, poisoning, and dependence. Pharmaceutical ads and warning labels try to help, but a recent analysis of U.S. advertising indicates many drugs are over-priced and positive effects are overemphasized. The accelerated use of direct-to-consumer ads even encourage patients to seek out more expensive and, often, inappropriate treatment. And then there are concerns about physician over-prescribing medications (e.g., Avitzur, 2014). To counterbalance these trends, workers have been typically educated on the risks and ways to make more informed choices. Apparently, this is not enough. We believe that workers now also need new, positive-focused, multi-pronged, and engaging well-being programs (based in research) that can train on positive mind-body habits. Such habits can mitigate against the need for drugs in the first place or help deter misuse (e.g., sharing drugs, non-medical use, accidental overdose).
Keywords. well-being programs, positive mind-body habits, workplace, integrated approach, Inappropriate Prescribing, Prescription Drug Misuse, Medication abuse, Work environment, Employee assistance programs, Well-being
Nasihati Gilani, S., Sheetz, K., Lucas, G. M., & Traum, D. (2016). What kind of stories should a virtual human swap? In Intelligent Virtual Agents: 16th International Conference, IVA 2016 (Lecture Notes in Computer Science, Vol. 10011) (pp. 128–140). Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-319-47665-0_12 Show abstract
Telling stories is an important aspect virtual agents designed to interact with people socially over time. We describe an experiment designed to investigate the impact of the identity, presentation form, and perspective of a virtual storyteller on a human user who engages in a story-swapping activity with two virtual characters. For each interaction, the user was given 10 “ice-breaker” questions to ask a virtual character and respond to the character’s reciprocal request. Participants also filled out a post-interaction survey, measuring rapport with the character and impressions of the character’s personality. Results generally show that participants prefer characters who tell first person stories, however there were some interactions with presentation order. No significant preferences were established for the form or identity variables.
Gratch, J., DeVault, D., & Lucas, G. M. (2016). The benefits of virtual humans for teaching negotiation. In Intelligent Virtual Agents: 16th International Conference, IVA 2016 (Lecture Notes in Computer Science, Vol. 10011) (pp. 283–294). Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-319-47665-0_25 Show abstract
This article examines the potential for teaching negotiation with virtual humans. Many people find negotiations to be aversive. We conjecture that students may be more comfortable practicing negotiation skills with an agent than with another person. We test this using the Conflict Resolution Agent, a semi-automated virtual human that negotiates with people via natural language. In a between-participants design, we independently manipulated two pedagogically-relevant factors while participants engaged in repeated negotiations with the agent: perceived agency (participants either believed they were negotiating with a computer program or another person) and pedagogical feedback (participants received instructional advice or no advice between negotiations). Findings indicate that novice negotiators were more comfortable negotiating with a computer program (they self-reported more comfort and punished their opponent less often) and expended more effort on the exercise following instructional feedback (both in time spent and in self-reported effort). These findings lend support to the notion of using virtual humans to teach interpersonal skills.
Lucas, G. M., Stratou, G., Lieblich, S., & Gratch, J. (2016). Trust me: Multimodal signals of trustworthiness. In Proceedings of the 18th ACM International Conference on Multimodal Interaction (ICMI '16) (pp. 5–12). Association for Computing Machinery. https://doi.org/10.1145/2993148.2993178 Show abstract
This paper builds on prior psychological studies that identify signals of trustworthiness between two human negotiators. Unlike prior work, the current work tracks such signals automatically and fuses them into computational models that predict trustworthiness. To achieve this goal, we apply automatic trackers to recordings of human dyads negotiating in a multi-issue bargaining task. We identify behavioral indicators in different modalities (facial expressions, gestures, gaze, and conversational features) that are predictive of trustworthiness. We predict both objective trustworthiness (i.e., are they honest) and perceived trustworthiness (i.e., do they seem honest to their interaction partner). Our experiments show that people are poor judges of objective trustworthiness (i.e., objective and perceived trustworthiness are predicted by different indicators), and that multimodal approaches better predict objective trustworthiness, whereas people overly rely on facial expressions when judging the honesty of their partner. Moreover, domain knowledge (from the literature and prior analysis of behaviors) facilitates the model development process.
Keywords. Negotiation, trustworthiness, automatic models for trustworthiness
Krämer, N. C., Karacora, B., Lucas, G. M., Dehghani, M., Rüther, G., & Gratch, J. (2016). Closing the gender gap in STEM with friendly male instructors? On the effects of rapport behavior and gender of a virtual agent in an instructional interaction. Computers & Education, 99, 1–13. https://doi.org/10.1016/j.compedu.2016.04.002 Show abstract
While numerous research endeavors address the effects of pedagogical agents, the role of the agent's gender and its rapport behavior has been neglected. We hypothesize that a minimal amount of behavioral realism induced by display of rapport is necessary for any social effects to occur in human-computer interaction. Further, in line with results from STEM research on female role models, we assume that especially for female learners a same sex agent will be beneficial. In a 2 (student gender) × 2 (agent gender) × 2 (rapport behavior yes/no) between subjects design, we investigate whether virtual agents can help enhance participants' performance, effort and motivation in mathematics. Female and male participants (N = 128) interacted with a male or female virtual agent that either displayed rapport or no rapport. Our results confirm the expected main effect of rapport. However, against expectations, our results do not support the assumption that a same sex agent is beneficial for female learners. Participants’ performance and effort were significantly enhanced when interacting with an agent of opposite gender that displayed rapport. Our results have implications on designing agents for education and training purposes.
Keywords. STEM; Rapport; Virtual agents; Gender; Performance in mathematics; Adult learning; Human-computer interface; Intelligent tutoring systems; Teaching/learning strategies
Nasihati Gilani, S., Sheetz, K., Lucas, G. M., & Traum, D. (2016). What kind of stories should a virtual human swap? [Extended abstract]. In Proceedings of the 15th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2016) (pp. 1437–1438). International Foundation for Autonomous Agents and Multiagent Systems. https://doi.org/10.65109/ksoy5146 Show abstract
Keywords. Virtual Humans, Story swapping, identity, user studies, rapport
Neeper, M., Bennett, J., Galvin, D., & Lucas, G. M. (2016). Workplace prevention of prescription drug abuse: Pilot assessment of a new psycho-educational program. Journal of Addiction Research & Therapy, 7(2), 277. https://doi.org/10.4172/2155-6105.1000277 Show abstract
Prescription drug misuse and abuse is a growing problem in America. It has negative effects on employees themselves, as well as productivity in the workplace, costing organizations money. This brief note describes a webinar that was designed to increase awareness of prescription drug abuse and misuse, as well as offering options on treatment for employees. 91% of participants agreed that they would use content from the webinar to assist the employees at their organizations. Ideally, a full pre-posttest analysis would be conducted on these data, but for now, these data provide support that the webinar was beneficial to participants.
Keywords. Prescription drug misuse, Abuse, Workplace, Employee assistance, EAP
Rizzo, A. A., Scherer, S., DeVault, D., Gratch, J., Artstein, R., Hartholt, A., Lucas, G. M., Marsella, S., Morbini, F., Nazarian, A., Stratou, G., Traum, D., Wood, R., Boberg, J., & Morency, L.-P. (2016). Detection and computational analysis of psychological signals using a virtual human interviewing agent. Journal of Pain Management, 9, 311–321. Full text Show abstract
It has long been recognized that facial expressions, body posture/gestures and vocal parameters play an important role in human communication and the implicit signalling of emotion. Recent advances in low cost computer vision and behavioral sensing technologies can now be applied to the process of making meaningful inferences as to user state when a person interacts with a computational device. Effective use of this additive information could serve to promote human interaction with virtual human (VH) agents that may enhance diagnostic assessment. This paper will focus on our current research in these areas within the DARPA-funded “Detection and Computational Analysis of Psychological Signals” project, with specific attention to the SimSensei application use case. SimSensei is a virtual human interaction platform that is able to sense and interpret real-time audiovisual behavioral signals from users interacting with the system. It is specifically designed for health care support and leverages years of virtual human research and development at USC-ICT. The platform enables an engaging face-to-face interaction where the virtual human automatically reacts to the state and inferred intent of the user through analysis of behavioral signals gleaned from facial expressions, body gestures and vocal parameters. Akin to how non-verbal behavioral signals have an impact on human to human interaction and communication, SimSensei aims to capture and infer from user non-verbal communication to improve engagement between a VH and a user. The system can also quantify and interpret sensed behavioral signals longitudinally that can be used to inform diagnostic assessment within a clinical context.
Keywords. Virtual human agents, sensing, computer vision, posttraumatic stress disorder
Wang, Y., Lucas, G. M., Khooshabeh, P., de Melo, C., & Gratch, J. (2015). Effects of emotional expressions on persuasion. Social Influence, 10(4), 236–249. https://doi.org/10.1080/15534510.2015.1081856 Show abstract
This paper investigates how expressions of emotion affect persuasiveness when the expresser and the recipient have different levels of power. The first study demonstrates that when the recipient overpowers the expresser, emotional expressions reduce persuasion. A second study reveals that power and perceived appropriateness of emotional expressions independently moderate the effect of emotional expressions. Emotional expressions hamper persuasion when the recipient overpowers the expresser, or when the emotional expressions are considered inappropriate.
Keywords. persuasion, emotion, expression, power, appropriateness
Nazari, Z., Lucas, G. M., & Gratch, J. (2015). Multimodal approach for automatic recognition of Machiavellianism. In 2015 International Conference on Affective Computing and Intelligent Interaction (ACII) (pp. 215–221). IEEE. https://doi.org/10.1109/ACII.2015.7344574 Show abstract
Machiavellianism, by definition, is the tendency to use other people as a tool to achieve one's own goals. Despite the large focus on the Big Five traits of personality, this anti-social trait is relatively unexplored in the computational realm. Automatically recognizing anti-social traits can have important uses across a variety of applications. In this paper, we use negotiation as a setting that provides Machiavellians with the opportunity to reveal their exploitative inclinations. We use textual, visual, acoustic, and behavioral cues to automatically predict High vs. Low Machiavellian personalities. These learned models have good accuracy when compared with other personality-recognition methods, and we provide evidence that the automatically-learned models are consistent with existing literature on this anti-social trait, giving evidence that these results can generalize to other domains.
Keywords. Machiavellianism, Classification, Personality, Dark Triad
Lucas, G. M., McCubbins, M. D., & Turner, M. (2015). Against game theory. In R. A. Scott & S. M. Kosslyn (Eds.), Emerging trends in the social and behavioral sciences: An interdisciplinary, searchable, and linkable resource (pp. 1–16). Hoboken, NJ: Wiley. https://doi.org/10.1002/9781118900772.etrds0004 Show abstract
People make choices. Often, the outcome depends on choices other people make. What mental steps do people go through when making such choices? Game theory, the most influential model of choice in economics and the social sciences, offers an answer, one based on games of strategy such as chess and checkers: the chooser considers the choices that others will make and makes a choice that will lead to a better outcome for the chooser, given all those choices by other people. It is universally established in the social sciences that classical game theory (even when heavily modified) is bad at predicting behavior. But instead of abandoning classical game theory, those in the social sciences have mounted a rescue operation under the name of “behavioral game theory.” Its main tool is to propose systematic deviations from the predictions of game theory, deviations that arise from character type, for example. Other deviations purportedly come from cognitive overload or limitations. The fundamental idea of behavioral game theory is that, if we know the deviations, then we can correct our predictions accordingly, and so get it right. There are two problems with this rescue operation, each of them is fatal. (i) For a chooser, contemplating the range of possible deviations, as there are many dozens, actually makes it exponentially harder to figure out a path to an outcome. This makes the theoretical models useless for modeling human thought or human behavior in general. (ii) Modeling deviations are helpful only if the deviations are consistent, so that scientists (and indeed decision makers) can make predictions about future choices on the basis of past choices. But the deviations are not consistent. In general, deviations from classical models are not consistent for any individual from one task to the next or between individuals for the same task. In addition, people's beliefs are in general not consistent with their choices. Accordingly, all hope is hollow that we can construct a general behavioral game theory. What can replace it? We survey some of the emerging candidates.
Scherer, S., Stratou, G., Lucas, G. M., Mahmoud, M., Boberg, J., Gratch, J., Rizzo, A., & Morency, L.-P. (2014). Automatic audiovisual behavior descriptors for psychological disorder analysis. Image and Vision Computing, 32(10), 648–658. https://doi.org/10.1016/j.imavis.2014.06.001 Show abstract
We investigate the capabilities of automatic audiovisual nonverbal behavior descriptors to identify indicators of psychological disorders such as depression, anxiety, and post-traumatic stress disorder. Due to strong correlations between these disorders as measured with standard self-assessment questionnaires in this study, we focus our investigations in particular on a generic distress measure as identified using factor analysis. Within this work, we seek to confirm and enrich present state of the art, predominantly based on qualitative manual annotations, with automatic quantitative behavior descriptors. We propose a number of nonverbal behavior descriptors that can be automatically estimated from audiovisual signals. Such automatic behavior descriptors could be used to support healthcare providers with quantified and objective observations that could ultimately improve clinical assessment. We evaluate our work on the dataset called the Distress Assessment Interview Corpus (DAIC) which comprises dyadic interactions between a confederate interviewer and a paid participant. Our evaluation on this dataset shows correlation of our automatic behavior descriptors with the derived general distress measure. Our analysis also includes a deeper study of self-adaptor and fidgeting behaviors based on detailed annotations of where these behaviors occur.
Keywords. Psychological distress, Depression, Post-traumatic stress disorder, Anxiety, Nonverbal behavior, Automatic assessment, Audiovisual
Lucas, G. M., Gratch, J., King, A., & Morency, L.-P. (2014). It’s only a computer: Virtual humans increase willingness to disclose. Computers in Human Behavior, 37, 94–100. https://doi.org/10.1016/j.chb.2014.04.043 Show abstract
Research has begun to explore the use of virtual humans (VHs) in clinical interviews (Bickmore, Gruber, & Picard, 2005). When designed as supportive and “safe” interaction partners, VHs may improve such screenings by increasing willingness to disclose information (Gratch, Wang, Gerten, & Fast, 2007). In health and mental health contexts, patients are often reluctant to respond honestly. In the context of health-screening interviews, we report a study in which participants interacted with a VH interviewer and were led to believe that the VH was controlled by either humans or automation. As predicted, compared to those who believed they were interacting with a human operator, participants who believed they were interacting with a computer reported lower fear of self-disclosure, lower impression management, displayed their sadness more intensely, and were rated by observers as more willing to disclose. These results suggest that automated VHs can help overcome a significant barrier to obtaining truthful patient information.
Keywords. Virtual humans, Computer-assisted assessment, Clinical interviews, Honest responding, Self-disclosure
Creativity and Innovation Award | 2025
Southern California Clinical and Translational Science Institute's Team Science Celebration Day
Artifact Award, Runner-up | 2025
Privacy Enhancing Technologies Symposium
Best Paper, Nomination | 2021
WeCNLP (West Coast NLP) Summit
Best Student Paper | 2021
Annual Conference on Advances in Cognitive Systems,
Winner of the Human-Agent League: "Pilot Agent" | 2020
Automated Negotiating Agents Competition, International Joint Conference on Artificial Intelligence
Service to the Conference | 2019
ACM International Conference on Intelligent User Interfaces
Best Socially Interactive Agents Paper, Nomination | 2018
Autonomous Agents and Multi-Agent Systems Conference
Best Paper | 2016
International Conference on Multimodal Interaction
Best Presentation | 2016
ACM SIGGRAPH Conference on Motion in Games
Best Paper | 2016
International Conference on Disability, Virtual Reality, and Associated Technologies
Best Poster | 2016
International Communication Association
Best Paper | 2015
Workshop on Automatic Sentiment Analysis in the Wild
Best Demo Paper | 2015
International Conference on Affective Computing and Intelligent Interaction
Best Paper, Runner-up | 2014
Autonomous Agents and Multi-Agent Systems Conference
Class of 1959 Distinguished Scholar Chair (inaugural recipient) | 2010
Willamette University,
Travel Award | 2007
Society for Personality and Social Psychology
Graduate Research Fellowship | 2006-2009
National Science Foundation
Teaching Assistant of the Year (inaugural recipient) | 2006
Northwestern University, Psychology Department
Fellowship | 2005-2006
Northwestern University
Graduated Summa Cum Laude and With Honors | 2005
Northwestern University
Noel F. Kaestner Psychology Award (for academic excellence & department service) | 2005
Willamette University
Presidential Senior Scholar Award | 2004-2005
Willamette University
Norman A. Huffman Award | 2004-2005
Willamette University
Phi Beta Kappa honor society (inducted as a junior) | 2004
Willamette University
Summer Science Fellow | 2003
American Psychological Association
G. Herbert Smith Presidential Award | 2001-2005
Willamette University
Remote Operated Construction Work: VR-based Training and Emerging Safety Needs
September 2026
–
August 2030
|
Active
Demolishing Barriers to Democratize Future Construction Operations by Providing Multi-sensory Capabilities for Effective Remote Work
October 2022
–
September 2027
|
Active
Detecting and mapping stress patterns across space and time: Multimodal modeling of individuals in real-world physical and social work environments
September 2022
–
February 2028
|
Co-Principal Investigator
|
Active
Co-adaptation of Intelligent Office Desks and Human Users to Promote Worker Productivity, Health and Wellness
August 2018
–
December 2022
|
Completed
⋯
Social
Facebook Instagram LinkedIn TikTok YouTube