Understanding Consent and Rights in AI Usage

If you are an organisation using AI or you are an AI developer, the Australian privacy regulator has just published some vital information about AI and your privacy obligations. Here is a summary of the new guides for businesses published today by the Office of the Australian Information Commissioner which articulate how Australian privacy law applies to AI and set out the regulator’s expectations. The first guide is aimed to help businesses comply with their privacy obligations when using commercially available AI products and help them to select an appropriate product. The second provides privacy guidance to developers using personal information to train generative AI models. GUIDE ONE: Guidance on privacy and the use of commercially available AI products Top five takeaways * Privacy obligations will apply to any personal information input into an AI system, as well as the output data generated by AI (where it contains personal information).  * Businesses should update their privacy policies and notifications with clear and transparent information about their use of AI * If AI systems are used to generate or infer personal information, including images, this is a collection of personal information and must comply with APP 3 (which deals with collection of personal info). * If personal information is being input into an AI system, APP 6 requires entities to only use or disclose the information for the primary purpose for which it was collected. * As a matter of best practice, the OAIC recommends that organisations do not enter personal information, and particularly sensitive information, into publicly available generative AI tools. GUIDE 2: Guidance on privacy and developing and training generative AI models Top five takeaways * Developers must take reasonable steps to ensure accuracy in generative AI models. * Just because data is publicly available or otherwise accessible does not mean it can legally be used to train or fine-tune generative AI models or systems.. * Developers must take particular care with sensitive information, which generally requires consent to be collected. * Where developers are seeking to use personal information that they already hold for the purpose of training an AI model, and this was not a primary purpose of collection, they need to carefully consider their privacy obligations. * Where a developer cannot clearly establish that a secondary use for an AI-related purpose was within reasonable expectations and related to a primary purpose, to avoid regulatory risk they should seek consent for that use and/or offer individuals a meaningful and informed ability to opt-out of such a use. https://lnkd.in/gX_FrtS9

The Oregon Department of Justice released new guidance on legal requirements when using AI. Here are the key privacy considerations, and four steps for companies to stay in-line with Oregon privacy law. ⤵️ The guidance details the AG's views of how uses of personal data in connection with AI or training AI models triggers obligations under the Oregon Consumer Privacy Act, including: 🔸Privacy Notices. Companies must disclose in their privacy notices when personal data is used to train AI systems. 🔸Consent. Updated privacy policies disclosing uses of personal data for AI training cannot justify the use of previously collected personal data for AI training; affirmative consent must be obtained. 🔸Revoking Consent. Where consent is provided to use personal data for AI training, there must be a way to withdraw consent and processing of that personal data must end within 15 days. 🔸Sensitive Data. Explicit consent must be obtained before sensitive personal data is used to develop or train AI systems. 🔸Training Datasets. Developers purchasing or using third-party personal data sets for model training may be personal data controllers, with all the required obligations that data controllers have under the law. 🔸Opt-Out Rights. Consumers have the right to opt-out of AI uses for certain decisions like housing, education, or lending. 🔸Deletion. Consumer #PersonalData deletion rights need to be respected when using AI models. 🔸Assessments. Using personal data in connection with AI models, or processing it in connection with AI models that involve profiling or other activities with heightened risk of harm, trigger data protection assessment requirements. The guidance also highlights a number of scenarios where sales practices using AI or misrepresentations due to AI use can violate the Unlawful Trade Practices Act. Here's a few steps to help stay on top of #privacy requirements under Oregon law and this guidance: 1️⃣ Confirm whether your organization or its vendors train #ArtificialIntelligence solutions on personal data.  2️⃣ Validate your organization's privacy notice discloses AI training practices. 3️⃣ Make sure organizational individual rights processes are scoped for personal data used in AI training. 4️⃣ Set assessment protocols where required to conduct and document data protection assessments that address the requirements under Oregon and other states' laws, and that are maintained in a format that can be provided to regulators.

At first glance, the Studio Ghibli style AI-generated art seems harmless. You upload a photo, the model processes it, and you get a stunning, anime-style transformation. But there's something far more complex beneath the surface—a quiet trade-off of identity, privacy, and control. Today, we casually give away fragments of ourselves: - Our faces to AI art apps - Our health data to wearables - Even our genetic blueprints to direct-to-consumer biotech services All in exchange for a few minutes of novelty or convenience. And while frameworks like India’s Digital Personal Data Protection Act (DPDPA) attempt to address this through “consent,” we must ask: What does consent even mean in an era of opaque AI systems designed to extract value far beyond that initial interaction? Because it’s not about the one image you uploaded. It’s about the aggregated behavioral and biometric insights these platforms derive from millions of us. That data trains models that can infer, profile, and yes—discriminate. Not just individually, but at community and population levels. This is no longer just a personal privacy issue. This is about digital sovereignty. Are we unintentionally allowing global AI systems to construct intimate, predictive bio-digital profiles of Indian citizens—only for that value to flow outward? And this isn’t just India’s challenge. Globally, these concerns resonate, creating complex challenges for cross-border data flows and requiring companies to navigate a patchwork of regulations like GDPR. The real risk isn’t that your selfie becomes a meme. It’s that your data contributes to shaping algorithms that may eventually determine what insurance you're offered, which job you’re filtered out of, or how your community is policed or advertised to, all without your knowledge or say. We need to go beyond checkbox consent. We need: 🔐 Privacy-by-design in every product 🛡️ Stronger enforcement of rights across borders 🧠 Collective awareness about how predictive analytics can influence entire societies Let’s be clear that innovation is critical. But if we don’t anchor it within ethics, rights, and sovereignty, we risk building tools that define and disadvantage us, rather than empower us. #Cybersecurity #PrivacyMatters #AIethics #DPDPA #DigitalSovereignty #DataProtection #AIresponsibility #IndiaTech

I've been beating this drum for a while now, but I think we're asking the wrong question about AI in therapy. "Should therapists use AI?" is a dead end. Technology isn't moral. It's just... there. Privacy disasters happen with plain old electronic health records, which means the tech itself has never been the point. The real question is: how do we make sure clients actually get to decide? The NASW Code of Ethics is pretty clear that client self-determination means therapists aren't the ultimate authority. So if we're introducing AI into someone's care, they should have a genuine, meaningful say in that. (Wild concept, I know.) Here's where I think we need to borrow from an unexpected place. Sexual ethics folks use "FRIES" to talk about consent—freely given, reversible, informed, enthusiastic, and specific. It maps surprisingly well onto what ethical AI consent should look like in therapy. Freely given means clients don't just... wake up with AI enabled because they missed a checkbox. The choice should be opt-in, presented separately from their therapist, probably during onboarding. Because asking someone to consent during a session is power dynamic problem. Reversible means people can change their minds. Maybe we check in monthly. Maybe before each session. The off switch should be obvious and easy. Informed means clients actually know what the AI does. What's recorded, who sees it, where it lives, who can access it. (At Allia Health, we're trying to figure out how to share this without overwhelming people who genuinely don't care about the details.) Enthusiastic means more than a rushed checkbox five minutes before your session starts. Specific means consent isn't a blanket "yes to whatever." Recording, summarizing, training future models, everything needs to be individually spelled out. And ideally, legally binding. I'm not here to take sides on whether AI belongs in therapy. But I am on the side of, "clients should be firmly in control of their own care". Tech will keep changing, and it might even make human-led therapy stronger. But we're not anyone's parents! It should always be up to the client.

A quality‑improvement study published in JAMA Network Open explores what matters when AI listens in on clinical encounters to generate documentation. This study focused on how informed consent is obtained. Highlights - Pilot across March–December 2024 in a large urban academic medical center - Involved 121 participants: 18 clinicians and 103 patients - Methodology included interviews, clinic observations, patient surveys, and clinician feedback to understand informed consent workflows Here's what they found... - The default consent approach was a verbal conversation between the clinician and the patient just before the visit - 74.8% of patients felt comfortable or very comfortable with ambient AI documentation - Crucially, comfort dropped when patients were disclosed complex technical details: *Basics only → 81.6% consented *Full disclosure of AI features, data storage, vendors → only 55.3% consented - Trust, clarity of discussion, and tool intent were key drivers of comfort and consent decisions - Perceived upsides included reduced admin work, better decision‑making, clearer patient–clinician dialogue - Concerns remained around data privacy, corporate liability, cognitive load, and equity - When asked about responsibility: *64.1% held physicians responsible for errors *76.7% held vendors responsible for breaches What patients and clinicians suggested – A flexible, multimodal consent model; combine verbal conversations, digital education, printed materials, staffed support, and signposted opt‑out options Dipu's Take: Ambient AI is accelerating clinician productivity, but consent frameworks must evolve in parallel. Even the best tools fail without human‑centered trust and transparent communication. https://lnkd.in/ehKSnSsV

UPDATE: On November 22, the update was added to the article basically saying that Google’s recent wording change around Gmail “smart features” caused major confusion — including early reports suggesting emails were being used to train Google’s AI models by default. After reviewing Google’s documentation, the author of the article concluded that “it doesn’t appear to be the case”. Gmail does scan content for built-in features like spam filtering and suggestions, but that is supposedly separate from training generative AI. 🤔 “… doesn’t appear to be the case” is the operative phrase in that update… Isn’t it? (Link to the updated source is in the comments). 🚨 Heads-up, cyber friends: your inbox might be humming with more than just deadlines. According to Malwarebytes, Gmail is automatically opting you in to have all your emails and attachments used for training its AI models. Unless you manually opt out, your private correspondence may now be fueling AI-features behind the scenes. Here are the key takeaways: 🔍 Opt-in by default matters — Instead of asking you first, the service assumes consent. This flips the script on personal privacy: it’s no longer “do you want to participate?” but “you are participating unless you act.” That shifts the power and — for many — erodes trust. 🤖 Training AI on consumer data without explicit consent is becoming a worrying trend. Using everyday user content (emails, attachments, chats) to refine AI models means personal information is being repurposed in unexpected ways. Even if anonymized, the fact that your private communications become a training set should raise eyebrows. 🛡️ Implications for professionals and individuals alike — If you handle sensitive info (clients, students, research, education), this isn’t just a nuisance; it’s a risk. Consent needs to be real, transparent and meaningful — not buried under settings toggles. 🧠 What you can do: Go into your Gmail settings, turn off “Smart features” in both Gmail/Chat/Meet and Workspace sections. Because yes, you have to flip both. In an era where data is called “the new oil,” assuming people want to pump their private life into AI-refineries without explicit agreement feels deeply off-brand for what privacy should mean. If we’re teaching the next generation how to think, how to work ethically, we can’t give tacit permission to a default that says “we’ll use your stuff unless you speak up.” As someone who lives at the intersection of cybersecurity, teaching, and digital citizenship, I say: We have to call this out. Let’s insist that “Yes” means yes, not “We quietly opted you in; you could opt out if you found it.” Control over personal data isn’t a bonus—it’s fundamental. #WomenInCyber #CyberSecurityLeadership #DataPrivacy #AIethics #ConsentFirst #StopAndSmellTheFlowers #ISSA #CyberThreatIntelligence #TechTrends #DigitalRights

📌 Automated Decision-Making under GDPR vs. CPRA: When Algorithms Decide Who Gets the Job - or the Loan AI doesn’t just predict anymore - it decides: who gets hired, insured, or approved for credit. Both GDPR and California’s CPRA have something to say about that. Let’s break it down 👇 🇪🇺 GDPR: Rights-Based and Restrictive GDPR doesn’t ban automated decisions outright - but it heavily regulates them when they significantly affect people. ✅ Article 22 gives individuals the right not to be subject to a decision based solely on automation that produces legal or similarly significant effects. 🧩 This means: - Individuals can request human review - Controllers must explain the logic and consequences - A legal basis is required — often explicit consent or contract - DPIAs are needed for high-risk profiling 🧪 Example: A fintech company in Germany uses AI to auto-approve loans. → That triggers Article 22 - requiring human review, transparency, and fairness. 💡 Bottom Line: GDPR keeps human oversight and rights at the center of algorithmic governance. 🇺🇸 CPRA: Rules Just Finalized California has now finalized its Automated Decision-Making Technology (ADMT) rules - adopted Oct 2025, effective Jan 2026. 🧭 The new regulations apply when AI replaces or substantially replaces human decision-making for key outcomes (jobs, credit, housing, health). They require: - Opt-out rights for certain ADMT uses - Disclosures explaining purpose, logic, and impacts - Risk assessments and cybersecurity audits 🧪 Example: A California retailer uses AI to screen job applications. → Under CPPA rules, it must disclose AI use, allow opt-outs, and document risk and fairness reviews. 💡 Bottom Line: California now has the first state-level AI governance framework directly addressing automated decisions. 🎯 The Core Difference GDPR → Rights-based regime - human review, transparency, and fairness required. CPRA (2026 +) → Governance-based regime - opt-outs, risk assessments, and disclosure duties driving accountability. 🌍 What This Says About Privacy Culture 🇪🇺 “We don’t let machines decide without a human in the loop.” 🇺🇸 “We’ll let them decide — but you’ll know how, and you can say no.” Same algorithm. Different philosophy. 👇 Coming soon: 🔹Cross-border transfers - and why EU → U.S. data flows still carry legal uncertainty? #GDPR #CPRA #AutomatedDecisionMaking #AICompliance #PrivacyLaw #CIPPUS #CIPPE #DataProtection #EUUSPrivacySeries #PrivacyProfessional #AIRegulation #LinkedInLearning #ADMT #GlobalPrivacy #InfoSec

Today, a recruiter invited me to a call about a potential role I was very interested in learning more about. But, less than an hour before the meeting, I received a sudden calendar update: “Fred from Fireflies will join to record and transcribe the conversation.” - No prior request for consent. - No explanation of how the recording would be stored. - No clear details on how my data might be used. What should have been a straightforward conversation instantly shifted into a scramble to protect my privacy (voice, image, and data). Recording an interview, without clear, advance permission, erodes trust before the first question is even asked. Consent is a deliberate agreement that lets everyone show up prepared and comfortable. This is an ethical issue. No doubt, an AI note-taker could be valuable to this recruiter. But, they also raise questions about data retention, confidentiality, and intellectual property. A candidate discussing career history, research, or sensitive client details deserves to know exactly how those records will be used and who will have access. If you truly aim to build an inclusive hiring process, plan for ethical recording practices from the first email. - State your intentions. - Outline how the file will be stored and data retention policies. - Offer alternative accommodations. - Secure explicit consent well before the call. Anything less feels like surveillance disguised as efficiency. How are you making sure your use of AI tools in interviews respects privacy, consent, and accessibility? *Note, I am fortunate to be able to walk away from situations that violate my privacy, and I did exactly that in this case. I recognize that many candidates cannot afford to decline and must navigate similar scenarios without the option to stay no. If you are in that position, I see you and stand with you. #CyberSecurity #DataPrivacy #Consent

NEVER use an AI notetaker in a meeting without obtaining explicit, informed consent from every participant ahead of time. 📝 This is an ethical imperative as AI becomes a standard part of many of our workflows. The primary consideration is not the utility of AI notetakers, but the privacy of others that must be respected when using them. Obtaining consent beforehand goes beyond procedural formality; it respects each individual's comfort, psychological safety, and right to choose. This explicit opt-in-only approach is crucial, especially in environments with inherent power dynamics. As a leader, always ensure that a team member knows without question that they can safely opt-out of AI being present in your meetings, with no strings attached (and no risk of punishment or damaging their relationship with you). ⛔It is NOT enough to ask for permission on the spot at the beginning of a meeting. It is essential to pre-communicate transparently with other meeting participants about your AI tool, its functionalities, and its data privacy implications. Such transparency allows participants the time and space to make informed decisions, and respects their right to set personal boundaries with this technology. This communication can also help preserve long-term professional relationships. If you ask on the spot (or worse, but increasingly common, don't ask at all), you put the other participants in a very uncomfortable position. You may suddenly find the person on the other end declining future meetings with you, and have no idea why (spoiler: it was your unethical use of AI!). Remember, even those who love and regularly use AI might have an issue with AI notetakers being present, so don't ever take it for granted that it's okay. If you want to use these tools, it's ALWAYS up to you to get consent, no matter who you're meeting with, not for anyone else to be in a position to say 'please get rid of it.' Respecting these nuances is vital for fostering a culture of mutual respect and ethical AI use, and ensuring you don't unwittingly damage your relationships. Let's all commit to setting the standard for AI notetaker use in our professional interactions, and make respecting everyone's right to choose whether an AI is recording them a professional and cultural norm. It starts with a simple yet powerful act: seeking informed consent well in advance. 🤔 Have you used AI notetakers in your meetings? How do you ensure you're respecting others when you do? Or maybe you've been in the position of AI joining without your permission? How did it make you feel? What did you do?

If you're deploying an AI scribe in clinical settings, a second class action just made the risk landscape a lot clearer. Three weeks ago, plaintiffs filed against Sutter Health and Memorial Health in federal court. Same theory as the Sharp HealthCare case from November. Same vendor. Same alleged violations — recording patient visits without proper consent, transmitting conversations to external servers, processing them through third-party AI. Two lawsuits in five months. Both in California. Both targeting the same ambient documentation technology. Here's what founders and operators need to understand: HIPAA compliance with your AI vendor is not enough. Both cases assert claims under California's wiretapping statutes and its medical privacy law — laws that impose separate consent requirements even when you have a valid BAA in place. In California, both the patient AND the clinician must consent before ambient recording begins. General privacy notices likely don't satisfy this. The more alarming allegation from Sharp: the AI system allegedly auto-inserted false consent statements into patient medical records — documenting that patients "were advised" and "consented" when they hadn't been told anything. If that allegation holds, you're looking at falsification of medical records at scale and all that means for patient safety and patient rights. It's bigger than just a privacy violation. And there's a new legal doctrine you need to know: the "capability test." A federal court ruled last year that an AI vendor can be liable as a third-party eavesdropper if it merely has the technical capability to use intercepted data for model training — regardless of whether it actually does. Check your terms of service. If it permits data use for product improvement, for example, and you don't have a plan for securing proper authorization, the exposure exists. If you're deploying ambient AI in clinical settings and want to talk through what your consent framework should look like, DM me or join the Elevare Law newsletter where I track this in real time.