InfoQ Homepage Prompt Engineering Content on InfoQ
News
RSS Feed-
How CyberArk Protects AI Agents with Instruction Detectors and History-Aware Validation
To prevent agents from obeying malicious instructions hidden in external data, all text entering an agent's context must be treated as untrusted, says Niv Rabin, principal software architect at AI-security firm CyberArk. His team developed an approach based on instruction detection and history-aware validation to protect against both malicious input data and context-history poisoning.
-
AI-Powered Code Editor Cursor Introduces Dynamic Context Discovery to Improve Token-Efficiency
Cursor introduced a new approach to minimize the context size of requests sent to large language models. Called dynamic context discovery, this method moves away from including large amounts of static context upfront, allowing the agent to dynamically retrieve only the information it needs. This reduces token usage and limits the inclusion of potentially confusing or irrelevant details.
-
AI Agents Fail Manipulation Tests in Microsoft's Magentic Marketplace Simulation
Researchers at Microsoft, working in collaboration with Arizona State University, have introduced Magentic Marketplace, an open-source simulation environment designed to study how LLM-based agents behave in multi-agent economic systems. The platform addresses a growing need in AI research as autonomous agents gain capabilities in software development.
-
How AI with Prompt Engineering Supports Software Testing
AI is becoming a key QA tool, aiding in faster scenario generation, risk detection, and test planning. Arbaz Surti showed how effective prompting using roles, context, and output format helps to get clear, relevant, and actionable test scenarios. AI can boost testers, but human judgment is needed to ensure relevance and quality.
-
OpenAI Launches Codex Software Engineering Agent Preview
OpenAI has launched Codex, a research preview of a cloud-based software engineering agent designed to automate common development tasks such as writing code, debugging, testing, and generating pull requests. Integrated into ChatGPT, Codex runs each assignment in a secure sandbox environment preloaded with the user's codebase and configured to reflect their development setup.
-
DeepMind Researchers Propose Defense against LLM Prompt Injection
To prevent prompt injection attacks when working with untrusted sources, Google DeepMind researchers have proposed CaMeL, a defense layer around LLMs that blocks malicious inputs by extracting the control and data flows from the query. According to their results, CaMeL can neutralize 67% of attacks in the AgentDojo security benchmark.
-
Google Gemini's Long-term Memory Vulnerable to a Kind of Phishing Attack
AI security hacker Johann Rehberger described a prompt injection attack against Google Gemini able to modify its long-term memories using a technique he calls delayed tool invocation. The researcher described the attack as a sort of social engineering/phishing attack triggered by the user interacting with a malicious document.
-
Google Introduces Veo and Imagen 3 for Advanced Media Generation on Vertex AI
Google Cloud has introduced Veo and Imagen 3, two new generative AI models available on its Vertex AI platform. Veo generates high-definition videos from text or image prompts, while Imagen 3 creates detailed, lifelike images. Both models include customization and editing tools, designed to support applications, with safety measures such as digital watermarking and data governance built-in.
-
QCon San Francisco 2024 Day 3: Arch Evolution, Next Gen UIs, Staff+ and Hardware Architectures
The 18th annual QCon San Francisco conference was held at the Hyatt Regency San Francisco in San Francisco, California. This five-day event, organized by C4Media, consists of three days of presentations and two days of workshops. Day Three, scheduled on November 20th, 2024, included two keynote addresses by Hien Luu and Shruti Bhat and presentations from four conference tracks.